Oct 30 03:01:33.112294 ip-10-0-132-171 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 30 03:01:33.112308 ip-10-0-132-171 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 30 03:01:33.112318 ip-10-0-132-171 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 30 03:01:33.112670 ip-10-0-132-171 systemd[1]: Failed to start Kubernetes Kubelet. Oct 30 03:01:42.684532 ip-10-0-132-171 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Oct 30 03:01:42.684545 ip-10-0-132-171 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 88063c7fb94f4f9aaeec4a9863035b1b -- Oct 30 03:03:34.325322 ip-10-0-132-171 systemd[1]: Starting Kubernetes Kubelet... Oct 30 03:03:34.774134 ip-10-0-132-171 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 03:03:34.774134 ip-10-0-132-171 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 03:03:34.774134 ip-10-0-132-171 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 03:03:34.774134 ip-10-0-132-171 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 03:03:34.779280 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779190 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:03:34.779280 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779274 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:03:34.779280 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779279 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:03:34.779280 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779284 2585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:03:34.779280 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779288 2585 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779293 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779296 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779301 2585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779305 2585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779309 2585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779313 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779316 2585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779320 2585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779331 2585 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779335 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779340 2585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779344 2585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779348 2585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779352 2585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779356 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779360 2585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779364 2585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779368 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:03:34.779580 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779372 2585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779376 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779380 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779384 2585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779388 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779392 2585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779396 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779400 2585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779404 2585 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779407 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779412 2585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779416 2585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779421 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779426 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779430 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779435 2585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779439 2585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779447 2585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779453 2585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779459 2585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:03:34.780347 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779463 2585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779468 2585 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779472 2585 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779477 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779481 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779485 2585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779489 2585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779494 2585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779498 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779502 2585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779506 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779510 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779515 2585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779519 2585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779523 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779528 2585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779532 2585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779537 2585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779541 2585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779546 2585 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:03:34.781236 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779552 2585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779556 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779561 2585 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779565 2585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779570 2585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779577 2585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779585 2585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779590 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779596 2585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779600 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779607 2585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779612 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779616 2585 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779621 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779625 2585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779629 2585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779634 2585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779638 2585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779643 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:03:34.782076 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779648 2585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779652 2585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779657 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779661 2585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779665 2585 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779669 2585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779674 2585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779678 2585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779682 2585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779687 2585 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779691 2585 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779696 2585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779701 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779705 2585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779709 2585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779714 2585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779718 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779723 2585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779727 2585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779732 2585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:03:34.782749 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779736 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779740 2585 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779745 2585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.779751 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781046 2585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781060 2585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781067 2585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781072 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781077 2585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781081 2585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781086 2585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781091 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781096 2585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781100 2585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781104 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781109 2585 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781113 2585 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781117 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781122 2585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:03:34.783226 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781127 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781131 2585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781136 2585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781140 2585 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781145 2585 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781151 2585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781155 2585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781160 2585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781164 2585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781168 2585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781173 2585 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781177 2585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781181 2585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781186 2585 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781192 2585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781198 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781203 2585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781208 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781214 2585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781218 2585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:03:34.783672 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781223 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781227 2585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781231 2585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781235 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781240 2585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781244 2585 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781248 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781252 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781256 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781261 2585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781266 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781271 2585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781275 2585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781280 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781284 2585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781289 2585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781293 2585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781297 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781302 2585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781306 2585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:03:34.784198 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781310 2585 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781315 2585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781319 2585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781323 2585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781328 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781332 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781336 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781340 2585 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781345 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781349 2585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781354 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781359 2585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781363 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781367 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781371 2585 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781375 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781380 2585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781384 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781388 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781392 2585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:03:34.784665 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781398 2585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781402 2585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781407 2585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781412 2585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781416 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781420 2585 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781424 2585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781428 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781432 2585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781437 2585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781441 2585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781445 2585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781449 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781454 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781458 2585 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781462 2585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781466 2585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781470 2585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781475 2585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:03:34.785155 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781479 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781484 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781488 2585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781492 2585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781496 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781501 2585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781505 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781510 2585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781514 2585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781518 2585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781523 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.781527 2585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782952 2585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782976 2585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782981 2585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782984 2585 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782987 2585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782990 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782993 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782996 2585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:03:34.786572 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.782999 2585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783002 2585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783005 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783008 2585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783010 2585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783013 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783016 2585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783019 2585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783021 2585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783024 2585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783027 2585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783030 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783032 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783035 2585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783038 2585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783040 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783043 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783046 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783048 2585 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783051 2585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:03:34.787080 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783054 2585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783056 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783059 2585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783061 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783068 2585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783072 2585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783075 2585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783078 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783080 2585 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783083 2585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783086 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783088 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783091 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783095 2585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783099 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783102 2585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783105 2585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783108 2585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783111 2585 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783114 2585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:03:34.787541 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783116 2585 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783119 2585 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783122 2585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783125 2585 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783127 2585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783131 2585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783133 2585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783136 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783139 2585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783142 2585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783145 2585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783148 2585 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783150 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783153 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783156 2585 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783159 2585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783161 2585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783164 2585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783167 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783170 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:03:34.788028 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783172 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783175 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783178 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783180 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783183 2585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783186 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783188 2585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783191 2585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783193 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783196 2585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783199 2585 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783201 2585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783204 2585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783206 2585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783209 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783212 2585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783214 2585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783217 2585 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783220 2585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:03:34.788490 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783225 2585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783228 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783231 2585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783234 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783236 2585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783239 2585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783242 2585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783244 2585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783247 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783250 2585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783254 2585 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783257 2585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783260 2585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783264 2585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783268 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783271 2585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783274 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783276 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.783279 2585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784436 2585 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 03:03:34.788929 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784446 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784452 2585 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784458 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784462 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784465 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784470 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784474 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784477 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784482 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784486 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784489 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784493 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784497 2585 flags.go:64] FLAG: --cgroup-root="" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784500 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784503 2585 flags.go:64] FLAG: --client-ca-file="" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784506 2585 flags.go:64] FLAG: --cloud-provider="external" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784509 2585 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784514 2585 flags.go:64] FLAG: --cluster-domain="" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784517 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784520 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784524 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784528 2585 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784532 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 03:03:34.789427 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784535 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784539 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784542 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784545 2585 flags.go:64] FLAG: --contention-profiling="false" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784548 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784551 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784554 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784557 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784562 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784565 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784568 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784571 2585 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784574 2585 flags.go:64] FLAG: --enable-server="true" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784577 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784582 2585 flags.go:64] FLAG: --event-burst="100" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784585 2585 flags.go:64] FLAG: --event-qps="50" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784588 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784591 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784595 2585 flags.go:64] FLAG: --eviction-hard="" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784599 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784603 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784606 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784609 2585 flags.go:64] FLAG: --eviction-soft="" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784612 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784615 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 03:03:34.790074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784619 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784622 2585 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784625 2585 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784628 2585 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784630 2585 flags.go:64] FLAG: --feature-gates="" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784635 2585 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784638 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784641 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784644 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784647 2585 flags.go:64] FLAG: --healthz-port="10248" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784650 2585 flags.go:64] FLAG: --help="false" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784654 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-132-171.ec2.internal" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784657 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784660 2585 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784663 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784666 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784670 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784673 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784676 2585 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784680 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784683 2585 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784686 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784689 2585 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784692 2585 flags.go:64] FLAG: --kube-reserved="" Oct 30 03:03:34.790726 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784695 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784698 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784701 2585 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784704 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784707 2585 flags.go:64] FLAG: --lock-file="" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784710 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784713 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784716 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784722 2585 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784725 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784728 2585 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784731 2585 flags.go:64] FLAG: --logging-format="text" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784734 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784738 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784741 2585 flags.go:64] FLAG: --manifest-url="" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784744 2585 flags.go:64] FLAG: --manifest-url-header="" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784749 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784752 2585 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784756 2585 flags.go:64] FLAG: --max-pods="110" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784760 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784763 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784765 2585 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784769 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784772 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784775 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 03:03:34.791354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784778 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784785 2585 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784788 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784791 2585 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784794 2585 flags.go:64] FLAG: --pod-cidr="" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784797 2585 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784800 2585 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784803 2585 flags.go:64] FLAG: --pods-per-core="0" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784806 2585 flags.go:64] FLAG: --port="10250" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784809 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784813 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fff32d287d94d358" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784816 2585 flags.go:64] FLAG: --qos-reserved="" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784819 2585 flags.go:64] FLAG: --read-only-port="10255" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784822 2585 flags.go:64] FLAG: --register-node="true" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784825 2585 flags.go:64] FLAG: --register-with-taints="" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784829 2585 flags.go:64] FLAG: --registry-burst="10" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784832 2585 flags.go:64] FLAG: --registry-qps="5" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784835 2585 flags.go:64] FLAG: --reserved-cpus="" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784838 2585 flags.go:64] FLAG: --reserved-memory="" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784842 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784845 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784848 2585 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784851 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784854 2585 flags.go:64] FLAG: --runonce="false" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784857 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784860 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784863 2585 flags.go:64] FLAG: --seccomp-default="false" Oct 30 03:03:34.791936 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784866 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784869 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784872 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784875 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784878 2585 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784881 2585 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784884 2585 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784889 2585 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784892 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784895 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784898 2585 flags.go:64] FLAG: --system-cgroups="" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784901 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784907 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784911 2585 flags.go:64] FLAG: --tls-cert-file="" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784914 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784918 2585 flags.go:64] FLAG: --tls-min-version="" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784922 2585 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784925 2585 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784928 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784931 2585 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784934 2585 flags.go:64] FLAG: --v="2" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784938 2585 flags.go:64] FLAG: --version="false" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784942 2585 flags.go:64] FLAG: --vmodule="" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784946 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.784950 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 03:03:34.792582 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785093 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785098 2585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785101 2585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785104 2585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785108 2585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785111 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785114 2585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785116 2585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785119 2585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785122 2585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785125 2585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785128 2585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785132 2585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785136 2585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785141 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785145 2585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785148 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785151 2585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785154 2585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:03:34.793186 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785157 2585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785161 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785164 2585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785173 2585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785176 2585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785179 2585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785182 2585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785185 2585 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785187 2585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785190 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785193 2585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785195 2585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785198 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785201 2585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785203 2585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785206 2585 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785209 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785211 2585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785214 2585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:03:34.793629 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785216 2585 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785219 2585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785222 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785224 2585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785227 2585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785230 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785232 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785235 2585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785239 2585 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785242 2585 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785244 2585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785247 2585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785250 2585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785253 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785257 2585 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785259 2585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785262 2585 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785265 2585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785268 2585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785271 2585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:03:34.794117 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785274 2585 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785276 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785279 2585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785282 2585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785284 2585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785287 2585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785290 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785292 2585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785295 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785298 2585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785300 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785303 2585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785306 2585 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785308 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785311 2585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785313 2585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785316 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785319 2585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785321 2585 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785324 2585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:03:34.794591 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785328 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785330 2585 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785333 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785336 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785339 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785341 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785346 2585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785350 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785353 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785356 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785358 2585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785361 2585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785364 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785367 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785369 2585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785372 2585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785375 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785377 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785380 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:03:34.795069 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785382 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785385 2585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785388 2585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785390 2585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785393 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785395 2585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785398 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785401 2585 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785403 2585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.785432 2585 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785471 2585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785477 2585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785480 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785485 2585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785489 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:03:34.795518 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785492 2585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785495 2585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785498 2585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785500 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785503 2585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785507 2585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785509 2585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785512 2585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785515 2585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785517 2585 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785520 2585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785523 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785526 2585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785529 2585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785531 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785534 2585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785536 2585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785539 2585 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785542 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:03:34.795879 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785544 2585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785547 2585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785550 2585 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785552 2585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785555 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785558 2585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785560 2585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785563 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785565 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785568 2585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785571 2585 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785575 2585 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785578 2585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785581 2585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785584 2585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785586 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785589 2585 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785591 2585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785595 2585 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785598 2585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:03:34.796337 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785601 2585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785603 2585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785606 2585 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785609 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785612 2585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785615 2585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785617 2585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785620 2585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785623 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785626 2585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785628 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785631 2585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785633 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785636 2585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785638 2585 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785641 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785644 2585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785646 2585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785649 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785652 2585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:03:34.796859 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785654 2585 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785657 2585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785660 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785663 2585 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785666 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785669 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785671 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785674 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785678 2585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785683 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785687 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785690 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785693 2585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785696 2585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785699 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785702 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785705 2585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785708 2585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785711 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785714 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:03:34.797373 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785717 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785719 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785722 2585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785724 2585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785727 2585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785730 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785732 2585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785735 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785738 2585 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785740 2585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785743 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785746 2585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785750 2585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785752 2585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785755 2585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785759 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785762 2585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785765 2585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785767 2585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:03:34.797952 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785770 2585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785773 2585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.785776 2585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.792778 2585 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.792795 2585 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.792812 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.792817 2585 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792870 2585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792874 2585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792878 2585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792881 2585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792885 2585 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792888 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792891 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792894 2585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792896 2585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792899 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792902 2585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792904 2585 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792907 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:03:34.798423 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792910 2585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792912 2585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792915 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792918 2585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792921 2585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792923 2585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792927 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792929 2585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792932 2585 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792935 2585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792938 2585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792940 2585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792943 2585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792945 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792948 2585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792951 2585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792955 2585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792958 2585 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792960 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:03:34.798933 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792979 2585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792986 2585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792990 2585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792993 2585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792997 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.792999 2585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793002 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793005 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793008 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793011 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793013 2585 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793016 2585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793019 2585 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793022 2585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793024 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793028 2585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793032 2585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793035 2585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793038 2585 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:03:34.799429 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793041 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793044 2585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793046 2585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793049 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793052 2585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793055 2585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793057 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793060 2585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793063 2585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793065 2585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793068 2585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793072 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793074 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793077 2585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793080 2585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793082 2585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793085 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793088 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793091 2585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793093 2585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:03:34.799909 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793096 2585 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793099 2585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793101 2585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793104 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793106 2585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793109 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793112 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793114 2585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793117 2585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793120 2585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793122 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793125 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793128 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793131 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793134 2585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793136 2585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793139 2585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793142 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793144 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:03:34.800389 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793147 2585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793150 2585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793152 2585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793156 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793158 2585 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793162 2585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793164 2585 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793167 2585 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793170 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793172 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793175 2585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793178 2585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793180 2585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793183 2585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793186 2585 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793188 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:03:34.800898 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.793216 2585 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793355 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793359 2585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793362 2585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793365 2585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793368 2585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793371 2585 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793374 2585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793376 2585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793380 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793382 2585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793385 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793388 2585 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793390 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793393 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793395 2585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793398 2585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793401 2585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793403 2585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:03:34.801300 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793406 2585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793408 2585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793411 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793418 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793421 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793423 2585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793426 2585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793428 2585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793431 2585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793434 2585 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793436 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793439 2585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793441 2585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793444 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793446 2585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793449 2585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793451 2585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793454 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793456 2585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:03:34.802047 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793459 2585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793462 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793464 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793467 2585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793470 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793472 2585 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793475 2585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793477 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793480 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793482 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793485 2585 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793488 2585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793491 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793493 2585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793496 2585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793498 2585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793501 2585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793504 2585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793507 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:03:34.802655 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793509 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793512 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793514 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793517 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793519 2585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793523 2585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793527 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793530 2585 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793533 2585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793535 2585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793538 2585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793540 2585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793543 2585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793546 2585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793548 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793551 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793553 2585 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793556 2585 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793558 2585 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793561 2585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:03:34.803149 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793563 2585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793566 2585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793569 2585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793571 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793574 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793577 2585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793579 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793582 2585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793584 2585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793587 2585 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793589 2585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793593 2585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793597 2585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793601 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793604 2585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793606 2585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793609 2585 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793611 2585 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793614 2585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:03:34.803669 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793616 2585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793619 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793621 2585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793624 2585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793627 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793629 2585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793632 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793635 2585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793638 2585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793641 2585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793644 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.793666 2585 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793709 2585 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793713 2585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793717 2585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:03:34.804227 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793721 2585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793725 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793728 2585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793731 2585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793734 2585 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793736 2585 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793739 2585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793741 2585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793744 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793747 2585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793750 2585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793752 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793755 2585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793758 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793761 2585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793763 2585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793766 2585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793768 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793771 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:03:34.804844 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793773 2585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793776 2585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793778 2585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793781 2585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793784 2585 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793787 2585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793789 2585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793792 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793795 2585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793798 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793800 2585 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793803 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793806 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793808 2585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793811 2585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793813 2585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793816 2585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793818 2585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793821 2585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:03:34.805359 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793823 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793826 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793828 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793831 2585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793834 2585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793837 2585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793839 2585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793842 2585 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793845 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793847 2585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793850 2585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793854 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793857 2585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793860 2585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793862 2585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793865 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793868 2585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793870 2585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793873 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:03:34.805794 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793875 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793878 2585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793880 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793883 2585 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793886 2585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793888 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793891 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793894 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793896 2585 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793899 2585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793902 2585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793904 2585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793906 2585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793909 2585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793911 2585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793914 2585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793916 2585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793919 2585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793922 2585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:03:34.806245 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793924 2585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793927 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793930 2585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793934 2585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793937 2585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793942 2585 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793945 2585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793948 2585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793951 2585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793955 2585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793957 2585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793960 2585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793977 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793980 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793984 2585 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793987 2585 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793989 2585 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793992 2585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793995 2585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.793998 2585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:03:34.806773 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.794001 2585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.794004 2585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.794007 2585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.794009 2585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.794012 2585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.794014 2585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:34.794017 2585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.794094 2585 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.796507 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.797579 2585 server.go:1014] "Starting client certificate rotation" Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.797682 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.797723 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.799812 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 03:03:34.807295 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.804237 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 03:03:34.814167 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.814149 2585 log.go:25] "Validated CRI v1 runtime API" Oct 30 03:03:34.818071 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.818055 2585 log.go:25] "Validated CRI v1 image API" Oct 30 03:03:34.818931 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.818909 2585 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 03:03:34.823163 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.823142 2585 fs.go:135] Filesystem UUIDs: map[09df95af-1b30-4872-915d-e67521ed2102:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 dcd426c2-9e85-4515-9106-9696fa7f0ea4:/dev/nvme0n1p3] Oct 30 03:03:34.823235 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.823161 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 03:03:34.828323 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.828220 2585 manager.go:217] Machine: {Timestamp:2026-10-30 03:03:34.826776193 +0000 UTC m=+0.371219731 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096710 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d8a0932c64a36ded936fc52f9c335 SystemUUID:ec2d8a09-32c6-4a36-ded9-36fc52f9c335 BootID:88063c7f-b94f-4f9a-aeec-4a9863035b1b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c6:37:e2:49 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c6:37:e2:49 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:75:1e:6f:e4:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 03:03:34.828323 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.828322 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 03:03:34.828436 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.828425 2585 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 03:03:34.829640 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.829616 2585 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 03:03:34.829762 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.829641 2585 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-171.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 03:03:34.829802 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.829764 2585 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 03:03:34.829802 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.829773 2585 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 03:03:34.829802 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.829789 2585 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 03:03:34.829802 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.829802 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 03:03:34.831094 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.831084 2585 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 03:03:34.835877 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.835865 2585 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 03:03:34.835997 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.835988 2585 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 03:03:34.836872 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.836863 2585 kubelet.go:493] "Attempting to sync node with API server" Oct 30 03:03:34.836906 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.836876 2585 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 03:03:34.836906 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.836886 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 03:03:34.836906 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.836895 2585 kubelet.go:405] "Adding apiserver pod source" Oct 30 03:03:34.836906 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.836903 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 03:03:34.837219 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.837204 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 03:03:34.839773 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.839736 2585 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 03:03:34.847742 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.847728 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 03:03:34.848612 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.848600 2585 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 03:03:34.848654 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.848630 2585 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 03:03:34.848776 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.848766 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 03:03:34.848814 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.848780 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 03:03:34.848814 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.848786 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 03:03:34.848814 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.848792 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 03:03:34.848814 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.848798 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 03:03:34.848814 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.848803 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 03:03:34.849490 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.849481 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 03:03:34.849490 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.849490 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 03:03:34.849549 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.849498 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 03:03:34.849549 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.849504 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 03:03:34.849549 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.849512 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 03:03:34.849549 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.849521 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 03:03:34.850789 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.850779 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 03:03:34.850832 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.850790 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 03:03:34.854633 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.854618 2585 server.go:1263] "Started kubelet" Oct 30 03:03:34.854765 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.854708 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 03:03:34.854851 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.854787 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 03:03:34.855394 ip-10-0-132-171 systemd[1]: Started Kubernetes Kubelet. Oct 30 03:03:34.858529 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.854748 2585 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 03:03:34.858861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.858843 2585 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 03:03:34.862617 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.862601 2585 server.go:317] "Adding debug handlers to kubelet server" Oct 30 03:03:34.865855 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.865835 2585 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 03:03:34.869279 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.869256 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 03:03:34.870642 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.870624 2585 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 03:03:34.870767 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.870733 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:34.871383 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871365 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 03:03:34.871383 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871368 2585 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 03:03:34.871501 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871377 2585 factory.go:55] Registering systemd factory Oct 30 03:03:34.871501 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871397 2585 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 03:03:34.871501 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871430 2585 factory.go:223] Registration of the systemd container factory successfully Oct 30 03:03:34.871501 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871496 2585 reconstruct.go:97] "Volume reconstruction finished" Oct 30 03:03:34.871642 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871507 2585 reconciler.go:29] "Reconciler: start to sync state" Oct 30 03:03:34.871679 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871641 2585 factory.go:153] Registering CRI-O factory Oct 30 03:03:34.871679 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871650 2585 factory.go:223] Registration of the crio container factory successfully Oct 30 03:03:34.871749 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871710 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 03:03:34.871749 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871727 2585 factory.go:103] Registering Raw factory Oct 30 03:03:34.871749 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.871741 2585 manager.go:1199] Started watching for new ooms in manager Oct 30 03:03:34.872821 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.872805 2585 manager.go:319] Starting recovery of all containers Oct 30 03:03:34.882775 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.882760 2585 manager.go:324] Recovery completed Oct 30 03:03:34.884362 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.883174 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-171.ec2.internal.18e32f99f1aac4aa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-171.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-171.ec2.internal,},FirstTimestamp:2026-10-30 03:03:34.854591658 +0000 UTC m=+0.399035196,LastTimestamp:2026-10-30 03:03:34.854591658 +0000 UTC m=+0.399035196,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-171.ec2.internal,}" Oct 30 03:03:34.884472 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.884452 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-171.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 03:03:34.888793 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.888774 2585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:03:34.890330 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.890314 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:03:34.890410 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.890341 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:03:34.890410 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.890352 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:03:34.890845 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.890828 2585 cpu_manager.go:226] "Starting" policy="none" Oct 30 03:03:34.890845 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.890843 2585 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 03:03:34.890947 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.890862 2585 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 03:03:34.890947 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.890830 2585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:03:34.891673 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.891657 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:03:34.891745 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.891683 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:03:34.891745 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.891695 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:03:34.893191 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.893175 2585 policy_none.go:50] "Start" Oct 30 03:03:34.893848 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.893836 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 03:03:34.893892 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.893855 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 03:03:34.895575 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.895558 2585 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 03:03:34.895895 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.895884 2585 policy_none.go:44] "Start" Oct 30 03:03:34.905210 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.905189 2585 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-171.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 03:03:34.905210 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.905190 2585 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 03:03:34.905486 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.905466 2585 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-171.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 03:03:34.905537 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.905460 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-171.ec2.internal.18e32f99f3cc14ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-171.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-171.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-171.ec2.internal,},FirstTimestamp:2026-10-30 03:03:34.890329262 +0000 UTC m=+0.434772800,LastTimestamp:2026-10-30 03:03:34.890329262 +0000 UTC m=+0.434772800,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-171.ec2.internal,}" Oct 30 03:03:34.927886 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.927811 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-171.ec2.internal.18e32f99f3cc5735 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-171.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-171.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-171.ec2.internal,},FirstTimestamp:2026-10-30 03:03:34.890346293 +0000 UTC m=+0.434789831,LastTimestamp:2026-10-30 03:03:34.890346293 +0000 UTC m=+0.434789831,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-171.ec2.internal,}" Oct 30 03:03:34.928028 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.928009 2585 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 03:03:34.932063 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.932039 2585 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 03:03:34.934325 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.934311 2585 manager.go:341] "Starting Device Plugin manager" Oct 30 03:03:34.935034 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.935016 2585 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 03:03:34.935034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.935031 2585 server.go:87] "Starting device plugin registration server" Oct 30 03:03:34.935210 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.935199 2585 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 03:03:34.935272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.935209 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 03:03:34.935497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.935306 2585 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 03:03:34.935497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.935383 2585 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 03:03:34.935497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.935391 2585 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 03:03:34.935837 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.935820 2585 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 03:03:34.935876 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.935857 2585 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:34.939802 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.939786 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zpsk8" Oct 30 03:03:34.953207 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.953128 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-171.ec2.internal.18e32f99f3cc7b8f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-171.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-171.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-171.ec2.internal,},FirstTimestamp:2026-10-30 03:03:34.890355599 +0000 UTC m=+0.434799137,LastTimestamp:2026-10-30 03:03:34.890355599 +0000 UTC m=+0.434799137,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-171.ec2.internal,}" Oct 30 03:03:34.969014 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:34.964870 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zpsk8" Oct 30 03:03:34.980571 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:34.980499 2585 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-171.ec2.internal.18e32f99f3cc14ae\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-171.ec2.internal.18e32f99f3cc14ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-171.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-171.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-171.ec2.internal,},FirstTimestamp:2026-10-30 03:03:34.890329262 +0000 UTC m=+0.434772800,LastTimestamp:2026-10-30 03:03:34.89167056 +0000 UTC m=+0.436114098,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-171.ec2.internal,}" Oct 30 03:03:35.009585 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.009561 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 03:03:35.010770 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.010752 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 03:03:35.010851 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.010776 2585 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 03:03:35.010851 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.010791 2585 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 03:03:35.010851 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.010828 2585 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 03:03:35.036047 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.035990 2585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:03:35.036243 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.036231 2585 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:03:35.036689 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.036676 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:03:35.036743 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.036700 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:03:35.036743 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.036710 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:03:35.036743 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.036730 2585 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.061131 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.061116 2585 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.061189 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.061135 2585 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-171.ec2.internal\": node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:35.105734 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.105715 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:35.111790 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.111771 2585 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal"] Oct 30 03:03:35.111861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.111831 2585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:03:35.112549 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.112535 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:03:35.112637 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.112563 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:03:35.112637 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.112580 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:03:35.114816 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.114802 2585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:03:35.114978 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.114955 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.115031 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.114999 2585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:03:35.115470 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.115456 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:03:35.115529 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.115480 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:03:35.115529 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.115495 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:03:35.115529 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.115510 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:03:35.115529 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.115526 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:03:35.115667 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.115538 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:03:35.117662 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.117650 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.117724 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.117669 2585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:03:35.118325 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.118310 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:03:35.118388 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.118334 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:03:35.118388 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.118349 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:03:35.158549 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.158529 2585 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-171.ec2.internal\" not found" node="ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.160599 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.160584 2585 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-171.ec2.internal\" not found" node="ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.206821 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.206803 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:35.272894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.272876 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.272960 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.272914 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.272960 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.272945 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e5cbe062ddc538d97b4fdc348b9dc76-config\") pod \"kube-apiserver-proxy-ip-10-0-132-171.ec2.internal\" (UID: \"0e5cbe062ddc538d97b4fdc348b9dc76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.307594 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.307550 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:35.374032 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.374011 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.374105 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.374042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.374105 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.374063 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e5cbe062ddc538d97b4fdc348b9dc76-config\") pod \"kube-apiserver-proxy-ip-10-0-132-171.ec2.internal\" (UID: \"0e5cbe062ddc538d97b4fdc348b9dc76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.374163 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.374115 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.374163 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.374124 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.374224 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.374115 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e5cbe062ddc538d97b4fdc348b9dc76-config\") pod \"kube-apiserver-proxy-ip-10-0-132-171.ec2.internal\" (UID: \"0e5cbe062ddc538d97b4fdc348b9dc76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.408525 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.408507 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:35.461477 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.461462 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.463630 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:35.463607 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:03:35.509256 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.509124 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:35.609795 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:35.609743 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:37.372786 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:37.372752 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:37.460283 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.460257 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 03:03:37.460722 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.460371 2585 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 03:03:37.473487 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:37.473469 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:37.573778 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:37.573754 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:37.583324 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.583299 2585 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:03:37.602119 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.602095 2585 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:03:37.631598 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:37.631563 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c72cf2f385a5a6399c0c2530d39d3b9.slice/crio-e694b13bb4c8ba60b7ab3c256513262d4a8cc5d4cc547880d9254dd9a9094d3d WatchSource:0}: Error finding container e694b13bb4c8ba60b7ab3c256513262d4a8cc5d4cc547880d9254dd9a9094d3d: Status 404 returned error can't find the container with id e694b13bb4c8ba60b7ab3c256513262d4a8cc5d4cc547880d9254dd9a9094d3d Oct 30 03:03:37.632683 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.632630 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 02:58:36 +0000 UTC" deadline="2028-05-17 15:25:38.34290523 +0000 UTC" Oct 30 03:03:37.632765 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.632686 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13572h22m0.710223269s" Oct 30 03:03:37.636128 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.636109 2585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:03:37.637103 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.637090 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:03:37.637153 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.637118 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:03:37.637153 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.637130 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:03:37.638495 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.638479 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 03:03:37.643692 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:37.642072 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e5cbe062ddc538d97b4fdc348b9dc76.slice/crio-74617c2f962a27e1da516a2cc4d931a206d42110349bc68969a6e6402fcaaaaa WatchSource:0}: Error finding container 74617c2f962a27e1da516a2cc4d931a206d42110349bc68969a6e6402fcaaaaa: Status 404 returned error can't find the container with id 74617c2f962a27e1da516a2cc4d931a206d42110349bc68969a6e6402fcaaaaa Oct 30 03:03:37.673912 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:37.673889 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:37.676754 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.676714 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" event={"ID":"0e5cbe062ddc538d97b4fdc348b9dc76","Type":"ContainerStarted","Data":"74617c2f962a27e1da516a2cc4d931a206d42110349bc68969a6e6402fcaaaaa"} Oct 30 03:03:37.677727 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:37.677709 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" event={"ID":"9c72cf2f385a5a6399c0c2530d39d3b9","Type":"ContainerStarted","Data":"e694b13bb4c8ba60b7ab3c256513262d4a8cc5d4cc547880d9254dd9a9094d3d"} Oct 30 03:03:37.774194 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:37.774176 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:37.874648 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:37.874601 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:37.975260 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:37.975236 2585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:03:38.033196 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.033174 2585 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:03:38.034165 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.034135 2585 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:03:38.058301 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.058277 2585 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 03:03:38.059409 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.059156 2585 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:03:38.082359 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.082241 2585 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 03:03:38.412403 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.412377 2585 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:03:38.501181 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.501157 2585 apiserver.go:52] "Watching apiserver" Oct 30 03:03:38.533169 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.533143 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 03:03:38.550958 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.550934 2585 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 03:03:38.551326 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.551302 2585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-8cffw","openshift-network-operator/iptables-alerter-klqwb","kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal","openshift-cluster-node-tuning-operator/tuned-jwjmm","openshift-dns/node-resolver-tslzq","openshift-multus/multus-additional-cni-plugins-wk25h","openshift-image-registry/node-ca-5pk5f","openshift-multus/multus-tbms7","openshift-ovn-kubernetes/ovnkube-node-lwxh9","kube-system/global-pull-secret-syncer-2p9zt","kube-system/konnectivity-agent-plrrl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal","openshift-multus/network-metrics-daemon-ks76k"] Oct 30 03:03:38.554075 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.554049 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.556178 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.556158 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:38.558372 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.558348 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:38.558472 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.558430 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:38.560550 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.560530 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.562838 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.562778 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.565076 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.565056 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.567753 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.567392 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.569774 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.569743 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.572778 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.572760 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 03:03:38.572884 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.572868 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-zrtj5\"" Oct 30 03:03:38.572953 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.572885 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rrhnz\"" Oct 30 03:03:38.573016 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.572968 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 03:03:38.573074 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.573016 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 03:03:38.573126 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.573071 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 03:03:38.573126 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.573097 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 03:03:38.575298 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.575283 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.577322 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.577305 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.577410 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.577375 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:38.580143 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.580125 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:38.580244 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.580201 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:38.582560 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.582542 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.595269 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.595245 2585 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 03:03:38.630847 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.630825 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dtcqt" Oct 30 03:03:38.635806 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.635790 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 03:03:38.645354 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645331 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-bin\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.645445 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645366 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a093097-53a0-40a5-9152-0e2930424cec-agent-certs\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:38.645445 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645389 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-modprobe-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.645445 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645405 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-socket-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.645445 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645427 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-daemon-config\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.645445 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645444 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-conf\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645460 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-lib-modules\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645474 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-tmp\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645489 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-system-cni-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645550 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-netns\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645569 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-systemd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645587 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-hostroot\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645611 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-systemd\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645633 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-var-lib-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645653 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-config\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-env-overrides\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645698 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovn-node-metrics-cert\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.645720 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645713 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-kubelet\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645728 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-dbus\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645749 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645765 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-cnibin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645784 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-multus\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645804 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-host\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645827 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nhkc\" (UniqueName: \"kubernetes.io/projected/31deab60-7ba7-431c-9fdd-f15388845315-kube-api-access-6nhkc\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645851 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-sys-fs\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645867 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-cni-binary-copy\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645886 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645902 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645916 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-netd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645937 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-device-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645951 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg72b\" (UniqueName: \"kubernetes.io/projected/1accce46-e302-4151-824a-5d9232a1be2b-kube-api-access-jg72b\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645969 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-socket-dir-parent\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.645989 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-conf-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646013 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-cnibin\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646029 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-bin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646044 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-etc-kubernetes\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646058 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh86z\" (UniqueName: \"kubernetes.io/projected/da1b6928-2507-4293-8e3c-f548c5b8ffab-kube-api-access-nh86z\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646072 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wptw5\" (UniqueName: \"kubernetes.io/projected/983ff918-e9a5-4966-ad38-7af3ae4a2bec-kube-api-access-wptw5\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646098 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn5xx\" (UniqueName: \"kubernetes.io/projected/0bb13fe7-8a37-41d7-90aa-fed271998088-kube-api-access-jn5xx\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646125 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-system-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646147 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-k8s-cni-cncf-io\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646169 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-host\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646191 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-netns\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646216 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-multus-certs\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646244 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-ovn\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646270 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646290 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgtzl\" (UniqueName: \"kubernetes.io/projected/59d544f8-cac5-4693-b82e-13f2ca910bfe-kube-api-access-sgtzl\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646307 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-kubelet-config\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646321 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-sys\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646337 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31deab60-7ba7-431c-9fdd-f15388845315-hosts-file\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.646861 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646356 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1b6928-2507-4293-8e3c-f548c5b8ffab-host-slash\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646381 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-etc-selinux\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646400 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646428 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-kubernetes\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646442 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-run\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646455 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-log-socket\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646473 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-script-lib\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646497 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g62l\" (UniqueName: \"kubernetes.io/projected/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-kube-api-access-6g62l\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646537 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da1b6928-2507-4293-8e3c-f548c5b8ffab-iptables-alerter-script\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646555 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-registration-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646570 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-serviceca\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646586 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hm8w\" (UniqueName: \"kubernetes.io/projected/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-kube-api-access-6hm8w\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646606 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646629 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646676 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-kubelet\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.647497 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646696 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-slash\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646728 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646752 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646774 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646806 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31deab60-7ba7-431c-9fdd-f15388845315-tmp-dir\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646822 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlncr\" (UniqueName: \"kubernetes.io/projected/f1078aa9-08f9-482d-b68c-16163edfaad8-kube-api-access-wlncr\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646838 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-etc-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646858 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-node-log\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646876 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-tuned\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646890 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-systemd-units\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646907 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a093097-53a0-40a5-9152-0e2930424cec-konnectivity-ca\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646934 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysconfig\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646959 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-var-lib-kubelet\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.648137 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.646978 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.651208 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.651191 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dtcqt" Oct 30 03:03:38.673202 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673158 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wmngf\"" Oct 30 03:03:38.673297 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673226 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 03:03:38.673297 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673264 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 03:03:38.673403 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673353 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 03:03:38.673403 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673387 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 03:03:38.673491 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673416 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 03:03:38.673567 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673490 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-676kj\"" Oct 30 03:03:38.673567 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673550 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 03:03:38.673661 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673553 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mhf7m\"" Oct 30 03:03:38.673661 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673586 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 03:03:38.673661 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673603 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 03:03:38.673661 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673613 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 03:03:38.673661 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673636 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 03:03:38.673661 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673641 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673770 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673787 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673804 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673806 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673828 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673831 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-mq4br\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673853 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673789 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673853 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673866 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5z89n\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673893 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h8g5n\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673898 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673902 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zjwd2\"" Oct 30 03:03:38.673934 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.673939 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 03:03:38.674569 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.674055 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.747898 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovn-node-metrics-cert\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.747937 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-kubelet\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.747965 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-dbus\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.747992 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748015 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-cnibin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748012 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-kubelet\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748037 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-multus\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748091 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-cnibin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748093 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-multus\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748111 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-host\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748137 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nhkc\" (UniqueName: \"kubernetes.io/projected/31deab60-7ba7-431c-9fdd-f15388845315-kube-api-access-6nhkc\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748159 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-sys-fs\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748182 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-cni-binary-copy\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748185 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748186 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-dbus\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748207 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748235 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-host\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.750034 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748256 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-sys-fs\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748294 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748302 2585 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-netd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748344 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-device-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748366 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg72b\" (UniqueName: \"kubernetes.io/projected/1accce46-e302-4151-824a-5d9232a1be2b-kube-api-access-jg72b\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748386 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-socket-dir-parent\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748386 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748407 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-conf-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748442 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-conf-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748445 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-cnibin\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748471 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-bin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748499 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-etc-kubernetes\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748502 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-netd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh86z\" (UniqueName: \"kubernetes.io/projected/da1b6928-2507-4293-8e3c-f548c5b8ffab-kube-api-access-nh86z\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748579 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wptw5\" (UniqueName: \"kubernetes.io/projected/983ff918-e9a5-4966-ad38-7af3ae4a2bec-kube-api-access-wptw5\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748605 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn5xx\" (UniqueName: \"kubernetes.io/projected/0bb13fe7-8a37-41d7-90aa-fed271998088-kube-api-access-jn5xx\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748640 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-system-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751048 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748667 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-k8s-cni-cncf-io\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-host\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748698 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-socket-dir-parent\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748717 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-netns\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748740 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-multus-certs\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748470 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-device-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-ovn\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748789 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-k8s-cni-cncf-io\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748789 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748738 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-bin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748796 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-etc-kubernetes\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748819 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgtzl\" (UniqueName: \"kubernetes.io/projected/59d544f8-cac5-4693-b82e-13f2ca910bfe-kube-api-access-sgtzl\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748830 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-multus-certs\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748852 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-kubelet-config\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748869 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-ovn\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748878 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-sys\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748766 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-cnibin\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748903 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31deab60-7ba7-431c-9fdd-f15388845315-hosts-file\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.751894 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748906 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-host\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748815 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-cni-binary-copy\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748943 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1b6928-2507-4293-8e3c-f548c5b8ffab-host-slash\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748950 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-netns\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748962 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31deab60-7ba7-431c-9fdd-f15388845315-hosts-file\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748969 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-etc-selinux\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748996 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749017 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-sys\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749031 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749044 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-system-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749052 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1b6928-2507-4293-8e3c-f548c5b8ffab-host-slash\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.748796 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749058 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-kubernetes\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749094 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-etc-selinux\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749096 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-kubelet-config\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749101 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-kubernetes\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749095 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-run\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749131 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-run\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.752717 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749133 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-log-socket\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749145 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749157 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-script-lib\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749181 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g62l\" (UniqueName: \"kubernetes.io/projected/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-kube-api-access-6g62l\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749187 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-log-socket\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.749183 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749206 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da1b6928-2507-4293-8e3c-f548c5b8ffab-iptables-alerter-script\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749229 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-registration-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.749276 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret podName:1bac3d0d-dded-4047-81d5-333e47612f93 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:39.249235572 +0000 UTC m=+3.131129744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret") pod "global-pull-secret-syncer-2p9zt" (UID: "1bac3d0d-dded-4047-81d5-333e47612f93") : object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749298 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-serviceca\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hm8w\" (UniqueName: \"kubernetes.io/projected/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-kube-api-access-6hm8w\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749348 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749371 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-kubelet\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749393 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-slash\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749417 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.753493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749494 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31deab60-7ba7-431c-9fdd-f15388845315-tmp-dir\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749731 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-script-lib\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749273 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-registration-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749796 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da1b6928-2507-4293-8e3c-f548c5b8ffab-iptables-alerter-script\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749837 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-slash\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749833 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31deab60-7ba7-431c-9fdd-f15388845315-tmp-dir\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749798 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749857 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-kubelet\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749912 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.749947 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749956 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-serviceca\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.749991 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs podName:59d544f8-cac5-4693-b82e-13f2ca910bfe nodeName:}" failed. No retries permitted until 2026-10-30 03:03:39.249975793 +0000 UTC m=+3.131869977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs") pod "network-metrics-daemon-ks76k" (UID: "59d544f8-cac5-4693-b82e-13f2ca910bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.749999 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750011 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlncr\" (UniqueName: \"kubernetes.io/projected/f1078aa9-08f9-482d-b68c-16163edfaad8-kube-api-access-wlncr\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-etc-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750066 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-node-log\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750088 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-tuned\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750110 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-systemd-units\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750112 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-etc-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750135 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a093097-53a0-40a5-9152-0e2930424cec-konnectivity-ca\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750159 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysconfig\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750182 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-var-lib-kubelet\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750203 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750219 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750227 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-bin\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750242 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-node-log\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750252 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a093097-53a0-40a5-9152-0e2930424cec-agent-certs\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750277 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-modprobe-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750285 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysconfig\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750287 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-systemd-units\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750299 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-socket-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750324 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-daemon-config\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.754869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750330 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-bin\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750359 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-conf\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750386 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-lib-modules\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750407 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-tmp\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750432 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-system-cni-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750456 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750479 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-netns\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750508 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-systemd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750550 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-hostroot\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750574 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-systemd\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750600 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-var-lib-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750626 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-config\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750649 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-env-overrides\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750751 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a093097-53a0-40a5-9152-0e2930424cec-konnectivity-ca\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750782 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-daemon-config\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750836 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-modprobe-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.750885 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-socket-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.755485 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751049 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751065 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-env-overrides\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751081 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-hostroot\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751102 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-var-lib-kubelet\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751132 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-systemd\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751144 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751150 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-systemd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751181 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-var-lib-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751222 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-netns\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751257 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-conf\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751298 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-system-cni-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751379 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-lib-modules\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.751640 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-config\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.753219 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-tuned\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.756346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.753392 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovn-node-metrics-cert\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.757055 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.756394 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a093097-53a0-40a5-9152-0e2930424cec-agent-certs\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:38.757465 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.757442 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-tmp\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.792935 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.792848 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlncr\" (UniqueName: \"kubernetes.io/projected/f1078aa9-08f9-482d-b68c-16163edfaad8-kube-api-access-wlncr\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.793740 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.793713 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg72b\" (UniqueName: \"kubernetes.io/projected/1accce46-e302-4151-824a-5d9232a1be2b-kube-api-access-jg72b\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.793740 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.793718 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nhkc\" (UniqueName: \"kubernetes.io/projected/31deab60-7ba7-431c-9fdd-f15388845315-kube-api-access-6nhkc\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.800841 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.800820 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 03:03:38.800952 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.800848 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 03:03:38.800952 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.800861 2585 projected.go:196] Error preparing data for projected volume kube-api-access-6bcnm for pod openshift-network-diagnostics/network-check-target-8cffw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:38.800952 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:38.800931 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm podName:be768a88-5dd3-44ff-800f-cd6603784ab4 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:39.300900719 +0000 UTC m=+3.182794910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6bcnm" (UniqueName: "kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm") pod "network-check-target-8cffw" (UID: "be768a88-5dd3-44ff-800f-cd6603784ab4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:38.803387 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.803344 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh86z\" (UniqueName: \"kubernetes.io/projected/da1b6928-2507-4293-8e3c-f548c5b8ffab-kube-api-access-nh86z\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.804250 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.804232 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hm8w\" (UniqueName: \"kubernetes.io/projected/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-kube-api-access-6hm8w\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.805165 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.805139 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn5xx\" (UniqueName: \"kubernetes.io/projected/0bb13fe7-8a37-41d7-90aa-fed271998088-kube-api-access-jn5xx\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.805289 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.805262 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wptw5\" (UniqueName: \"kubernetes.io/projected/983ff918-e9a5-4966-ad38-7af3ae4a2bec-kube-api-access-wptw5\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.806913 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.806894 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g62l\" (UniqueName: \"kubernetes.io/projected/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-kube-api-access-6g62l\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.808406 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.808361 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgtzl\" (UniqueName: \"kubernetes.io/projected/59d544f8-cac5-4693-b82e-13f2ca910bfe-kube-api-access-sgtzl\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:38.868983 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.868949 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:03:38.878417 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.878392 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:38.885001 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.884980 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:03:38.892065 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.892044 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tslzq" Oct 30 03:03:38.896191 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.896174 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a093097_53a0_40a5_9152_0e2930424cec.slice/crio-143ae3ac6af31cbca158f8457cd5debdc7ef2a07b7c508d9dcd71ef1378691bf WatchSource:0}: Error finding container 143ae3ac6af31cbca158f8457cd5debdc7ef2a07b7c508d9dcd71ef1378691bf: Status 404 returned error can't find the container with id 143ae3ac6af31cbca158f8457cd5debdc7ef2a07b7c508d9dcd71ef1378691bf Oct 30 03:03:38.897596 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.897558 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod206d6d3e_a3c9_44ff_80b7_bbc97ab9fc45.slice/crio-14ef22c78b45926184f1ff02ee96b2def3755aaea1a9e1114f1c7e284fa44d1a WatchSource:0}: Error finding container 14ef22c78b45926184f1ff02ee96b2def3755aaea1a9e1114f1c7e284fa44d1a: Status 404 returned error can't find the container with id 14ef22c78b45926184f1ff02ee96b2def3755aaea1a9e1114f1c7e284fa44d1a Oct 30 03:03:38.900653 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.900629 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983ff918_e9a5_4966_ad38_7af3ae4a2bec.slice/crio-7ee93d3958e623cbab1df65cd14151d1680d3605649ddff0cab62149f4d55fd1 WatchSource:0}: Error finding container 7ee93d3958e623cbab1df65cd14151d1680d3605649ddff0cab62149f4d55fd1: Status 404 returned error can't find the container with id 7ee93d3958e623cbab1df65cd14151d1680d3605649ddff0cab62149f4d55fd1 Oct 30 03:03:38.907467 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.907443 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31deab60_7ba7_431c_9fdd_f15388845315.slice/crio-9c8f3d51d1799a83759828681971124ce841d8a032e9f04d9497febb5277a8ff WatchSource:0}: Error finding container 9c8f3d51d1799a83759828681971124ce841d8a032e9f04d9497febb5277a8ff: Status 404 returned error can't find the container with id 9c8f3d51d1799a83759828681971124ce841d8a032e9f04d9497febb5277a8ff Oct 30 03:03:38.918290 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.918272 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:03:38.920698 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.920671 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tbms7" Oct 30 03:03:38.929809 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.929786 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:03:38.931068 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.931047 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bb13fe7_8a37_41d7_90aa_fed271998088.slice/crio-5b7cfbc5319e7f442eea768b44a14b523b9e5edfd84298635a40d58ca3b39671 WatchSource:0}: Error finding container 5b7cfbc5319e7f442eea768b44a14b523b9e5edfd84298635a40d58ca3b39671: Status 404 returned error can't find the container with id 5b7cfbc5319e7f442eea768b44a14b523b9e5edfd84298635a40d58ca3b39671 Oct 30 03:03:38.931963 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.931942 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1078aa9_08f9_482d_b68c_16163edfaad8.slice/crio-168dd325ea4c992d8739825170c9cabf4d9d930c68c308b377d787bc1b159b57 WatchSource:0}: Error finding container 168dd325ea4c992d8739825170c9cabf4d9d930c68c308b377d787bc1b159b57: Status 404 returned error can't find the container with id 168dd325ea4c992d8739825170c9cabf4d9d930c68c308b377d787bc1b159b57 Oct 30 03:03:38.939587 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.939567 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:03:38.939874 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.939852 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1b6928_2507_4293_8e3c_f548c5b8ffab.slice/crio-e4e7ca17a2a73713d132705c2942e4e37f78afab4691539f23103ded1c329067 WatchSource:0}: Error finding container e4e7ca17a2a73713d132705c2942e4e37f78afab4691539f23103ded1c329067: Status 404 returned error can't find the container with id e4e7ca17a2a73713d132705c2942e4e37f78afab4691539f23103ded1c329067 Oct 30 03:03:38.941747 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:38.941728 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:38.952044 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.952010 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1accce46_e302_4151_824a_5d9232a1be2b.slice/crio-56a70dd10da311078ba4997ed73aeda78169deebedf9de4447bccda37f4d2d5a WatchSource:0}: Error finding container 56a70dd10da311078ba4997ed73aeda78169deebedf9de4447bccda37f4d2d5a: Status 404 returned error can't find the container with id 56a70dd10da311078ba4997ed73aeda78169deebedf9de4447bccda37f4d2d5a Oct 30 03:03:38.955056 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:03:38.954821 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9057bfb2_faf8_4a2b_bca6_3f1d5ab89f24.slice/crio-f5c361011544f41d06155e7697e42b2f1385e67addaff7ba4584c0ae6bff14f5 WatchSource:0}: Error finding container f5c361011544f41d06155e7697e42b2f1385e67addaff7ba4584c0ae6bff14f5: Status 404 returned error can't find the container with id f5c361011544f41d06155e7697e42b2f1385e67addaff7ba4584c0ae6bff14f5 Oct 30 03:03:39.254597 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.254132 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:39.254597 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.254185 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:39.254597 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:39.254306 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:39.254597 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:39.254358 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs podName:59d544f8-cac5-4693-b82e-13f2ca910bfe nodeName:}" failed. No retries permitted until 2026-10-30 03:03:40.254342162 +0000 UTC m=+4.136236345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs") pod "network-metrics-daemon-ks76k" (UID: "59d544f8-cac5-4693-b82e-13f2ca910bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:39.254597 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:39.254415 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:39.254597 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:39.254446 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret podName:1bac3d0d-dded-4047-81d5-333e47612f93 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:40.254436181 +0000 UTC m=+4.136330355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret") pod "global-pull-secret-syncer-2p9zt" (UID: "1bac3d0d-dded-4047-81d5-333e47612f93") : object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:39.355355 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.355113 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:39.355543 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:39.355326 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 03:03:39.355543 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:39.355439 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 03:03:39.355543 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:39.355455 2585 projected.go:196] Error preparing data for projected volume kube-api-access-6bcnm for pod openshift-network-diagnostics/network-check-target-8cffw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:39.355543 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:39.355528 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm podName:be768a88-5dd3-44ff-800f-cd6603784ab4 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:40.355495929 +0000 UTC m=+4.237390100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bcnm" (UniqueName: "kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm") pod "network-check-target-8cffw" (UID: "be768a88-5dd3-44ff-800f-cd6603784ab4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:39.660319 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.654318 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 02:58:38 +0000 UTC" deadline="2028-07-10 07:08:43.805295834 +0000 UTC" Oct 30 03:03:39.660319 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.654374 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14860h5m4.150925704s" Oct 30 03:03:39.700131 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.697732 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" event={"ID":"1accce46-e302-4151-824a-5d9232a1be2b","Type":"ContainerStarted","Data":"56a70dd10da311078ba4997ed73aeda78169deebedf9de4447bccda37f4d2d5a"} Oct 30 03:03:39.709899 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.709869 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tbms7" event={"ID":"0bb13fe7-8a37-41d7-90aa-fed271998088","Type":"ContainerStarted","Data":"5b7cfbc5319e7f442eea768b44a14b523b9e5edfd84298635a40d58ca3b39671"} Oct 30 03:03:39.714756 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.714722 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tslzq" event={"ID":"31deab60-7ba7-431c-9fdd-f15388845315","Type":"ContainerStarted","Data":"9c8f3d51d1799a83759828681971124ce841d8a032e9f04d9497febb5277a8ff"} Oct 30 03:03:39.718342 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.718273 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" event={"ID":"983ff918-e9a5-4966-ad38-7af3ae4a2bec","Type":"ContainerStarted","Data":"7ee93d3958e623cbab1df65cd14151d1680d3605649ddff0cab62149f4d55fd1"} Oct 30 03:03:39.727274 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.727247 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5pk5f" event={"ID":"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45","Type":"ContainerStarted","Data":"14ef22c78b45926184f1ff02ee96b2def3755aaea1a9e1114f1c7e284fa44d1a"} Oct 30 03:03:39.737896 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.737868 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-klqwb" event={"ID":"da1b6928-2507-4293-8e3c-f548c5b8ffab","Type":"ContainerStarted","Data":"e4e7ca17a2a73713d132705c2942e4e37f78afab4691539f23103ded1c329067"} Oct 30 03:03:39.741699 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.741636 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerStarted","Data":"168dd325ea4c992d8739825170c9cabf4d9d930c68c308b377d787bc1b159b57"} Oct 30 03:03:39.751195 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.751171 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-plrrl" event={"ID":"9a093097-53a0-40a5-9152-0e2930424cec","Type":"ContainerStarted","Data":"143ae3ac6af31cbca158f8457cd5debdc7ef2a07b7c508d9dcd71ef1378691bf"} Oct 30 03:03:39.756540 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.755882 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" event={"ID":"0e5cbe062ddc538d97b4fdc348b9dc76","Type":"ContainerStarted","Data":"285df8f8d4e3e831df9470a15075191991b81948bbd017b973b7327b673e0afa"} Oct 30 03:03:39.769320 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.769295 2585 generic.go:356] "Generic (PLEG): container finished" podID="9c72cf2f385a5a6399c0c2530d39d3b9" containerID="e158f69ccd1227e481c49b96e4423015e1f06845604968a0ba6955ae0710f9ef" exitCode=0 Oct 30 03:03:39.769426 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.769370 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" event={"ID":"9c72cf2f385a5a6399c0c2530d39d3b9","Type":"ContainerDied","Data":"e158f69ccd1227e481c49b96e4423015e1f06845604968a0ba6955ae0710f9ef"} Oct 30 03:03:39.783066 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.782960 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"f5c361011544f41d06155e7697e42b2f1385e67addaff7ba4584c0ae6bff14f5"} Oct 30 03:03:39.859184 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:39.859084 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" podStartSLOduration=1.859064358 podStartE2EDuration="1.859064358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:39.810433772 +0000 UTC m=+3.692327964" watchObservedRunningTime="2026-10-30 03:03:39.859064358 +0000 UTC m=+3.740958550" Oct 30 03:03:40.262974 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.262210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:40.262974 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.262264 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:40.262974 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.262398 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:40.262974 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.262460 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs podName:59d544f8-cac5-4693-b82e-13f2ca910bfe nodeName:}" failed. No retries permitted until 2026-10-30 03:03:42.262440174 +0000 UTC m=+6.144334363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs") pod "network-metrics-daemon-ks76k" (UID: "59d544f8-cac5-4693-b82e-13f2ca910bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:40.262974 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.262844 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:40.262974 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.262896 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret podName:1bac3d0d-dded-4047-81d5-333e47612f93 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:42.262880122 +0000 UTC m=+6.144774309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret") pod "global-pull-secret-syncer-2p9zt" (UID: "1bac3d0d-dded-4047-81d5-333e47612f93") : object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:40.363299 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.362747 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:40.363299 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.362912 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 03:03:40.363299 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.362929 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 03:03:40.363299 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.362940 2585 projected.go:196] Error preparing data for projected volume kube-api-access-6bcnm for pod openshift-network-diagnostics/network-check-target-8cffw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:40.363299 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.362993 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm podName:be768a88-5dd3-44ff-800f-cd6603784ab4 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:42.362977538 +0000 UTC m=+6.244871722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bcnm" (UniqueName: "kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm") pod "network-check-target-8cffw" (UID: "be768a88-5dd3-44ff-800f-cd6603784ab4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:40.654915 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.654820 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 02:58:38 +0000 UTC" deadline="2028-08-10 00:00:05.938828608 +0000 UTC" Oct 30 03:03:40.654915 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.654886 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15596h56m25.283946556s" Oct 30 03:03:40.675373 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.674671 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:40.675373 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.674700 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:40.675373 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.674796 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:40.675373 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.675007 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:40.675373 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.675141 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:40.675373 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:40.675230 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:40.792775 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.792740 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" event={"ID":"9c72cf2f385a5a6399c0c2530d39d3b9","Type":"ContainerStarted","Data":"fd8848ad4841b5d861aa42812a14653e7a81579079340eee7820c3fbd55cb2b6"} Oct 30 03:03:40.851525 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:40.851319 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" podStartSLOduration=2.8513000059999998 podStartE2EDuration="2.851300006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:40.850307432 +0000 UTC m=+4.732201625" watchObservedRunningTime="2026-10-30 03:03:40.851300006 +0000 UTC m=+4.733194197" Oct 30 03:03:41.799312 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:41.799219 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerStarted","Data":"1be8f45d0b2e5bd05ac7bd6b684f6f0d62ad4309d0d556d580991f1314f3b8e0"} Oct 30 03:03:42.279297 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.279263 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:42.279481 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.279320 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:42.279642 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.279490 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:42.279642 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.279562 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs podName:59d544f8-cac5-4693-b82e-13f2ca910bfe nodeName:}" failed. No retries permitted until 2026-10-30 03:03:46.279544628 +0000 UTC m=+10.161438802 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs") pod "network-metrics-daemon-ks76k" (UID: "59d544f8-cac5-4693-b82e-13f2ca910bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:42.279978 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.279955 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:42.279978 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.280016 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret podName:1bac3d0d-dded-4047-81d5-333e47612f93 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:46.280001657 +0000 UTC m=+10.161895849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret") pod "global-pull-secret-syncer-2p9zt" (UID: "1bac3d0d-dded-4047-81d5-333e47612f93") : object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:42.380486 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.379899 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:42.380486 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.380037 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 03:03:42.380486 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.380057 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 03:03:42.380486 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.380071 2585 projected.go:196] Error preparing data for projected volume kube-api-access-6bcnm for pod openshift-network-diagnostics/network-check-target-8cffw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:42.380486 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.380130 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm podName:be768a88-5dd3-44ff-800f-cd6603784ab4 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:46.380110846 +0000 UTC m=+10.262005020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bcnm" (UniqueName: "kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm") pod "network-check-target-8cffw" (UID: "be768a88-5dd3-44ff-800f-cd6603784ab4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:42.676832 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.676054 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:42.676832 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.676179 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:42.676832 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.676510 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:42.676832 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.676621 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:42.676832 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.676664 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:42.676832 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:42.676742 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:42.808009 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.807969 2585 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="1be8f45d0b2e5bd05ac7bd6b684f6f0d62ad4309d0d556d580991f1314f3b8e0" exitCode=0 Oct 30 03:03:42.812780 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.812687 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerDied","Data":"1be8f45d0b2e5bd05ac7bd6b684f6f0d62ad4309d0d556d580991f1314f3b8e0"} Oct 30 03:03:42.825127 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:42.825094 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-plrrl" event={"ID":"9a093097-53a0-40a5-9152-0e2930424cec","Type":"ContainerStarted","Data":"a92574295685c25113aa1efff1141b6a98ebf52b5b6bbff4de49e504fe5aba2d"} Oct 30 03:03:43.837657 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:43.837577 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5pk5f" event={"ID":"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45","Type":"ContainerStarted","Data":"50901d35cd247aa6624227bf6f812acb3cdcc301dff8de5e0d1236e7eb590983"} Oct 30 03:03:43.890310 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:43.890254 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-plrrl" podStartSLOduration=4.384700524 podStartE2EDuration="7.890238256s" totalImagesPullingTime="3.505537732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:42.906363656 +0000 UTC m=+6.788257848" watchObservedRunningTime="2026-10-30 03:03:43.890238256 +0000 UTC m=+7.772132472" Oct 30 03:03:44.674711 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:44.674632 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:44.674866 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:44.674755 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:44.675085 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:44.675067 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:44.675169 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:44.675154 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:44.675231 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:44.675206 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:44.675298 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:44.675279 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:44.841836 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:44.841602 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" event={"ID":"1accce46-e302-4151-824a-5d9232a1be2b","Type":"ContainerStarted","Data":"7e298a1fe74f56a3c0c98b486e6d5de4e774880bd3687fea1c828ab5f3cc545e"} Oct 30 03:03:45.947701 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:45.947674 2585 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 03:03:46.084364 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.084197 2585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:46.085127 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.085065 2585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:46.128251 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.128196 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5pk5f" podStartSLOduration=5.453166151 podStartE2EDuration="10.128180088s" totalImagesPullingTime="4.675013937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:43.890905115 +0000 UTC m=+7.772799307" watchObservedRunningTime="2026-10-30 03:03:46.128180088 +0000 UTC m=+10.010074280" Oct 30 03:03:46.318881 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.318500 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:46.318881 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.318658 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:46.318881 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.318676 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:46.318881 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.318734 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret podName:1bac3d0d-dded-4047-81d5-333e47612f93 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:54.318714255 +0000 UTC m=+18.200608447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret") pod "global-pull-secret-syncer-2p9zt" (UID: "1bac3d0d-dded-4047-81d5-333e47612f93") : object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:46.318881 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.318804 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:46.318881 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.318855 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs podName:59d544f8-cac5-4693-b82e-13f2ca910bfe nodeName:}" failed. No retries permitted until 2026-10-30 03:03:54.318840505 +0000 UTC m=+18.200734691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs") pod "network-metrics-daemon-ks76k" (UID: "59d544f8-cac5-4693-b82e-13f2ca910bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:46.419891 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.419854 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:46.420077 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.420057 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 03:03:46.420151 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.420085 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 03:03:46.420151 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.420098 2585 projected.go:196] Error preparing data for projected volume kube-api-access-6bcnm for pod openshift-network-diagnostics/network-check-target-8cffw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:46.420220 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.420153 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm podName:be768a88-5dd3-44ff-800f-cd6603784ab4 nodeName:}" failed. No retries permitted until 2026-10-30 03:03:54.420135032 +0000 UTC m=+18.302029220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bcnm" (UniqueName: "kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm") pod "network-check-target-8cffw" (UID: "be768a88-5dd3-44ff-800f-cd6603784ab4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:46.608221 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.607590 2585 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T03:03:45.94769533Z","UUID":"0af85544-15bd-4664-a5f0-70c4d6752722","Handler":null,"Name":"","Endpoint":""} Oct 30 03:03:46.609290 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.609253 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 03:03:46.609482 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.609443 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 03:03:46.676401 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.676377 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:46.676546 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.676463 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:46.676546 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.676509 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:46.676621 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.676598 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:46.676655 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.676637 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:46.676695 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:46.676679 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:46.853472 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.853395 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" event={"ID":"1accce46-e302-4151-824a-5d9232a1be2b","Type":"ContainerStarted","Data":"efe622a1d0b002694806e4dff96d9935485ce085a274d6f2b74272df266828d0"} Oct 30 03:03:46.856209 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.855473 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tbms7" event={"ID":"0bb13fe7-8a37-41d7-90aa-fed271998088","Type":"ContainerStarted","Data":"df8e22ccdc830ba9c608cbb92cda9631e5e311f0692e66e53572d5f83e194113"} Oct 30 03:03:46.859279 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.859259 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tslzq" event={"ID":"31deab60-7ba7-431c-9fdd-f15388845315","Type":"ContainerStarted","Data":"b39a1a3144569008e5b9c7e84a8a3c521306adc85f2d5ce1906ead3de602e792"} Oct 30 03:03:46.907759 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.907714 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-tbms7" podStartSLOduration=3.741836215 podStartE2EDuration="10.907699134s" totalImagesPullingTime="7.165862919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:46.906927939 +0000 UTC m=+10.788822131" watchObservedRunningTime="2026-10-30 03:03:46.907699134 +0000 UTC m=+10.789593335" Oct 30 03:03:46.950602 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:46.950378 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-tslzq" podStartSLOduration=3.259244108 podStartE2EDuration="10.950363159s" totalImagesPullingTime="7.691119051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:46.950274056 +0000 UTC m=+10.832168250" watchObservedRunningTime="2026-10-30 03:03:46.950363159 +0000 UTC m=+10.832257351" Oct 30 03:03:47.863687 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:47.863648 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" event={"ID":"1accce46-e302-4151-824a-5d9232a1be2b","Type":"ContainerStarted","Data":"a1c35dc694abd935a9811cf565dba2d5547822a8d92613ae370716f55197de95"} Oct 30 03:03:47.865877 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:47.865816 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-klqwb" event={"ID":"da1b6928-2507-4293-8e3c-f548c5b8ffab","Type":"ContainerStarted","Data":"0b65c3bf0ed12d060f5e619b3a6b4a14ebc04c0e0e52589d7ca5c9b0147d6d02"} Oct 30 03:03:47.868278 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:47.868256 2585 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="5e970f7ea1f2ab88b086aa640472f9da5f8a245629100b0fa65186dbf58b217b" exitCode=0 Oct 30 03:03:47.868400 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:47.868330 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerDied","Data":"5e970f7ea1f2ab88b086aa640472f9da5f8a245629100b0fa65186dbf58b217b"} Oct 30 03:03:47.912147 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:47.912093 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" podStartSLOduration=4.200611254 podStartE2EDuration="11.912072897s" totalImagesPullingTime="7.711461643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:47.911038688 +0000 UTC m=+11.792932879" watchObservedRunningTime="2026-10-30 03:03:47.912072897 +0000 UTC m=+11.793967090" Oct 30 03:03:48.006306 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:48.006218 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-klqwb" podStartSLOduration=4.614208581 podStartE2EDuration="12.006200342s" totalImagesPullingTime="7.391991761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:48.005631099 +0000 UTC m=+11.887525291" watchObservedRunningTime="2026-10-30 03:03:48.006200342 +0000 UTC m=+11.888094536" Oct 30 03:03:48.673648 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:48.673617 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:48.673648 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:48.673644 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:48.673887 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:48.673753 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:48.673887 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:48.673834 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:48.673986 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:48.673906 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:48.673986 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:48.673967 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:49.877443 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:49.877408 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" event={"ID":"983ff918-e9a5-4966-ad38-7af3ae4a2bec","Type":"ContainerStarted","Data":"eea472c5e87454b639bd0d94372ecf8e64cda928fc84979ee2a4b9719a8d8143"} Oct 30 03:03:49.880068 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:49.880034 2585 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="61488809824a71c8b859f9dc7b6ef7acdbff57888ba5298e6bbbcf770ff7afef" exitCode=0 Oct 30 03:03:49.880208 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:49.880090 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerDied","Data":"61488809824a71c8b859f9dc7b6ef7acdbff57888ba5298e6bbbcf770ff7afef"} Oct 30 03:03:49.928170 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:49.928119 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" podStartSLOduration=4.001459345 podStartE2EDuration="13.928103803s" totalImagesPullingTime="9.926644458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:49.927725398 +0000 UTC m=+13.809619617" watchObservedRunningTime="2026-10-30 03:03:49.928103803 +0000 UTC m=+13.809997991" Oct 30 03:03:50.677900 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:50.677871 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:50.678063 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:50.677906 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:50.678063 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:50.677980 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:50.678063 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:50.677994 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:50.678208 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:50.678080 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:50.678208 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:50.678175 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:50.885810 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:50.885574 2585 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="882f7bb8a0b3344702988bbc7b31b8440fc60bed93532ca48a9ef2f02d71f989" exitCode=0 Oct 30 03:03:50.886214 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:50.885636 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerDied","Data":"882f7bb8a0b3344702988bbc7b31b8440fc60bed93532ca48a9ef2f02d71f989"} Oct 30 03:03:52.673566 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:52.673533 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:52.674010 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:52.673531 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:52.674010 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:52.673659 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:52.674010 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:52.673673 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:52.674010 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:52.673793 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:52.674010 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:52.673885 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:53.894615 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.894384 2585 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="8d4a1595f67e6bbdaeeb2d0e098c0f0d132aca11f400c3f99fc401cb99c76122" exitCode=0 Oct 30 03:03:53.895083 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.894467 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerDied","Data":"8d4a1595f67e6bbdaeeb2d0e098c0f0d132aca11f400c3f99fc401cb99c76122"} Oct 30 03:03:53.897680 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.897658 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"486782b0cdedfd7cafda7eeed62aec8af11510d68c666cadc14d800504a4f533"} Oct 30 03:03:53.897745 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.897714 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"db4b7715d2fd3a8629e8e980b543a9084016ab3afaad7d8139500838d5d6c928"} Oct 30 03:03:53.897776 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.897743 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"d7a077d798cc976bff34a3f2e2cac81f278db99ae4fa8e2eee7de037754c3473"} Oct 30 03:03:53.897776 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.897770 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"026d2417215bb8e96976dcb3d685e3a60c9ca35710f1a2a49e65b2baffcd8131"} Oct 30 03:03:53.897837 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.897805 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"e59ce98b1d914899a2ed7d77d446f6f7d7f451f17d06bb59c7af559b0bf121d5"} Oct 30 03:03:53.897872 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.897838 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"139723e6129be35991295a68a396ff803b19c9cae280e27b60aa322da2587f77"} Oct 30 03:03:53.971131 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.971104 2585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:53.971222 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.971205 2585 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 03:03:53.971716 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:53.971695 2585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:03:54.377086 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:54.377055 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:54.377086 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:54.377094 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:54.377296 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.377203 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:54.377296 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.377255 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:54.377296 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.377261 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret podName:1bac3d0d-dded-4047-81d5-333e47612f93 nodeName:}" failed. No retries permitted until 2026-10-30 03:04:10.377247627 +0000 UTC m=+34.259141809 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret") pod "global-pull-secret-syncer-2p9zt" (UID: "1bac3d0d-dded-4047-81d5-333e47612f93") : object "kube-system"/"original-pull-secret" not registered Oct 30 03:03:54.377296 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.377292 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs podName:59d544f8-cac5-4693-b82e-13f2ca910bfe nodeName:}" failed. No retries permitted until 2026-10-30 03:04:10.377281147 +0000 UTC m=+34.259175332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs") pod "network-metrics-daemon-ks76k" (UID: "59d544f8-cac5-4693-b82e-13f2ca910bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 03:03:54.478052 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:54.478013 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:54.478181 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.478137 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 03:03:54.478181 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.478153 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 03:03:54.478181 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.478161 2585 projected.go:196] Error preparing data for projected volume kube-api-access-6bcnm for pod openshift-network-diagnostics/network-check-target-8cffw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:54.478293 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.478205 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm podName:be768a88-5dd3-44ff-800f-cd6603784ab4 nodeName:}" failed. No retries permitted until 2026-10-30 03:04:10.478193876 +0000 UTC m=+34.360088058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bcnm" (UniqueName: "kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm") pod "network-check-target-8cffw" (UID: "be768a88-5dd3-44ff-800f-cd6603784ab4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 03:03:54.674252 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:54.674171 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:54.674252 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:54.674192 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:54.674440 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.674288 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:54.674440 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.674353 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:54.674440 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:54.674371 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:54.674606 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:54.674447 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:54.904422 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:54.904394 2585 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="1f22696ed015bf99bf1e9eb1d8af91d259dc3a0074638c6684f89e65a60104c3" exitCode=0 Oct 30 03:03:54.904869 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:54.904477 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerDied","Data":"1f22696ed015bf99bf1e9eb1d8af91d259dc3a0074638c6684f89e65a60104c3"} Oct 30 03:03:55.909710 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:55.909615 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"74e0fcceeba72e06230f699f3828b2a292d86d9f6eeea2b52483a430b64e2f3c"} Oct 30 03:03:55.912743 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:55.912714 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wk25h" event={"ID":"f1078aa9-08f9-482d-b68c-16163edfaad8","Type":"ContainerStarted","Data":"8893d42e8a0d310af95d6529cce4f9c337e35e96c34a7cb6187785d765638f57"} Oct 30 03:03:55.962300 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:55.962261 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wk25h" podStartSLOduration=9.289764846 podStartE2EDuration="19.962249225s" totalImagesPullingTime="10.672484379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:55.961934585 +0000 UTC m=+19.843828776" watchObservedRunningTime="2026-10-30 03:03:55.962249225 +0000 UTC m=+19.844143415" Oct 30 03:03:56.674511 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:56.674480 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:56.674667 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:56.674609 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:56.674667 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:56.674614 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:56.674767 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:56.674738 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:56.674809 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:56.674782 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:56.674878 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:56.674860 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:57.921182 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:57.921149 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" event={"ID":"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24","Type":"ContainerStarted","Data":"3db52b6ce34104a80bdb278436fdaeb71f888cbc2b939a89ca8efa3fd0d9ddcc"} Oct 30 03:03:57.921708 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:57.921375 2585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:57.921708 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:57.921454 2585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:57.936774 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:57.936703 2585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:58.016786 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:58.016399 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" podStartSLOduration=7.773260104 podStartE2EDuration="22.016383675s" totalImagesPullingTime="14.243123571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:03:58.015452275 +0000 UTC m=+21.897346462" watchObservedRunningTime="2026-10-30 03:03:58.016383675 +0000 UTC m=+21.898277866" Oct 30 03:03:58.674090 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:58.674062 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:58.674090 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:58.674089 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:58.674284 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:58.674162 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:58.674322 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:58.674278 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:58.674404 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:58.674387 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:03:58.674454 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:58.674445 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:58.926403 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:58.925113 2585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:58.944303 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:58.944274 2585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:03:59.761626 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:59.761563 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2p9zt"] Oct 30 03:03:59.761770 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:59.761661 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:03:59.761770 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:59.761738 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:03:59.764534 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:59.764498 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8cffw"] Oct 30 03:03:59.764639 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:59.764612 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:03:59.764714 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:59.764699 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:03:59.764977 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:59.764958 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ks76k"] Oct 30 03:03:59.765050 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:03:59.765040 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:03:59.765128 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:03:59.765113 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:04:01.673989 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:01.673949 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:01.673989 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:01.673997 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:01.673989 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:01.673987 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:01.674537 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:01.674074 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:04:01.674537 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:01.674121 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:04:01.674537 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:01.674217 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:04:03.673964 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:03.673923 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:03.674402 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:03.674014 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:03.674402 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:03.674023 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:04:03.674402 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:03.674105 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:03.674402 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:03.674099 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:04:03.674402 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:03.674183 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:04:05.674036 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:05.674008 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:05.674456 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:05.674107 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:05.674456 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:05.674117 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:05.674456 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:05.674118 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2p9zt" podUID="1bac3d0d-dded-4047-81d5-333e47612f93" Oct 30 03:04:05.674456 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:05.674189 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ks76k" podUID="59d544f8-cac5-4693-b82e-13f2ca910bfe" Oct 30 03:04:05.674456 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:05.674286 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cffw" podUID="be768a88-5dd3-44ff-800f-cd6603784ab4" Oct 30 03:04:06.977670 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:06.977481 2585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeReady" Oct 30 03:04:06.978028 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:06.977755 2585 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 03:04:07.084039 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.084012 2585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b4nrx"] Oct 30 03:04:07.164681 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.164648 2585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nxn6c"] Oct 30 03:04:07.164834 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.164767 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:07.178097 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.178073 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b4nrx"] Oct 30 03:04:07.178097 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.178103 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nxn6c"] Oct 30 03:04:07.178238 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.178204 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.181740 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.181718 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 03:04:07.181859 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.181806 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-5jrfh\"" Oct 30 03:04:07.181914 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.181882 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 03:04:07.181914 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.181889 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 03:04:07.193409 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.193392 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jsmnt\"" Oct 30 03:04:07.193555 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.193540 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 03:04:07.193596 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.193567 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 03:04:07.276593 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.276547 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5170510-91aa-451e-bc7d-e18c3aaca68b-tmp-dir\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.276593 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.276574 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlj67\" (UniqueName: \"kubernetes.io/projected/bdb6477c-e16b-451c-84bd-8e62d3efc79f-kube-api-access-xlj67\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:07.276593 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.276597 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.276785 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.276665 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlprd\" (UniqueName: \"kubernetes.io/projected/d5170510-91aa-451e-bc7d-e18c3aaca68b-kube-api-access-rlprd\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.276785 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.276712 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdb6477c-e16b-451c-84bd-8e62d3efc79f-cert\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:07.276785 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.276767 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5170510-91aa-451e-bc7d-e18c3aaca68b-config-volume\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.377334 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.377311 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5170510-91aa-451e-bc7d-e18c3aaca68b-tmp-dir\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.377418 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.377343 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlj67\" (UniqueName: \"kubernetes.io/projected/bdb6477c-e16b-451c-84bd-8e62d3efc79f-kube-api-access-xlj67\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:07.377418 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.377361 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.377540 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:07.377464 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 03:04:07.377540 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.377490 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlprd\" (UniqueName: \"kubernetes.io/projected/d5170510-91aa-451e-bc7d-e18c3aaca68b-kube-api-access-rlprd\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.377540 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:07.377534 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls podName:d5170510-91aa-451e-bc7d-e18c3aaca68b nodeName:}" failed. No retries permitted until 2026-10-30 03:04:07.877499011 +0000 UTC m=+31.759393184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls") pod "dns-default-nxn6c" (UID: "d5170510-91aa-451e-bc7d-e18c3aaca68b") : secret "dns-default-metrics-tls" not found Oct 30 03:04:07.377697 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.377561 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdb6477c-e16b-451c-84bd-8e62d3efc79f-cert\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:07.377697 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.377625 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5170510-91aa-451e-bc7d-e18c3aaca68b-config-volume\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.377769 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.377697 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5170510-91aa-451e-bc7d-e18c3aaca68b-tmp-dir\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.378031 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.378008 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5170510-91aa-451e-bc7d-e18c3aaca68b-config-volume\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.381383 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.381357 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdb6477c-e16b-451c-84bd-8e62d3efc79f-cert\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:07.402350 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.402324 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlprd\" (UniqueName: \"kubernetes.io/projected/d5170510-91aa-451e-bc7d-e18c3aaca68b-kube-api-access-rlprd\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.402612 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.402598 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlj67\" (UniqueName: \"kubernetes.io/projected/bdb6477c-e16b-451c-84bd-8e62d3efc79f-kube-api-access-xlj67\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:07.486329 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.486303 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:07.674346 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.674320 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:07.674474 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.674323 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:07.674543 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.674336 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:07.692345 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.692329 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 03:04:07.692484 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.692472 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 03:04:07.692639 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.692620 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 03:04:07.692683 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.692654 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qvz6h\"" Oct 30 03:04:07.692737 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.692725 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lplr6\"" Oct 30 03:04:07.692784 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.692770 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nxm67\"" Oct 30 03:04:07.692892 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.692874 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 03:04:07.847635 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.847608 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b4nrx"] Oct 30 03:04:07.848815 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:04:07.848786 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb6477c_e16b_451c_84bd_8e62d3efc79f.slice/crio-eccd84d9e3c2a2ab6e44722edbc11c8f09d729fcfcaf3d9dc7e3756ac70a2943 WatchSource:0}: Error finding container eccd84d9e3c2a2ab6e44722edbc11c8f09d729fcfcaf3d9dc7e3756ac70a2943: Status 404 returned error can't find the container with id eccd84d9e3c2a2ab6e44722edbc11c8f09d729fcfcaf3d9dc7e3756ac70a2943 Oct 30 03:04:07.881447 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.881423 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:07.881612 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:07.881595 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 03:04:07.881686 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:07.881662 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls podName:d5170510-91aa-451e-bc7d-e18c3aaca68b nodeName:}" failed. No retries permitted until 2026-10-30 03:04:08.881642934 +0000 UTC m=+32.763537117 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls") pod "dns-default-nxn6c" (UID: "d5170510-91aa-451e-bc7d-e18c3aaca68b") : secret "dns-default-metrics-tls" not found Oct 30 03:04:07.942460 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:07.942432 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b4nrx" event={"ID":"bdb6477c-e16b-451c-84bd-8e62d3efc79f","Type":"ContainerStarted","Data":"eccd84d9e3c2a2ab6e44722edbc11c8f09d729fcfcaf3d9dc7e3756ac70a2943"} Oct 30 03:04:08.888856 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:08.888820 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:08.889299 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:08.888995 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 03:04:08.889299 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:08.889061 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls podName:d5170510-91aa-451e-bc7d-e18c3aaca68b nodeName:}" failed. No retries permitted until 2026-10-30 03:04:10.88904844 +0000 UTC m=+34.770942611 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls") pod "dns-default-nxn6c" (UID: "d5170510-91aa-451e-bc7d-e18c3aaca68b") : secret "dns-default-metrics-tls" not found Oct 30 03:04:09.498703 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.498673 2585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2"] Oct 30 03:04:09.517288 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.517259 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" Oct 30 03:04:09.524732 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.524703 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2"] Oct 30 03:04:09.540194 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.540151 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 03:04:09.540309 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.540270 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 03:04:09.551466 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.551444 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-g6lgw\"" Oct 30 03:04:09.595318 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.595286 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zrc6\" (UniqueName: \"kubernetes.io/projected/662545a9-fe29-4583-8fdd-8ce384ee6681-kube-api-access-2zrc6\") pod \"migrator-66df5d496b-pdqh2\" (UID: \"662545a9-fe29-4583-8fdd-8ce384ee6681\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" Oct 30 03:04:09.696574 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.696537 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zrc6\" (UniqueName: \"kubernetes.io/projected/662545a9-fe29-4583-8fdd-8ce384ee6681-kube-api-access-2zrc6\") pod \"migrator-66df5d496b-pdqh2\" (UID: \"662545a9-fe29-4583-8fdd-8ce384ee6681\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" Oct 30 03:04:09.726929 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.726900 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zrc6\" (UniqueName: \"kubernetes.io/projected/662545a9-fe29-4583-8fdd-8ce384ee6681-kube-api-access-2zrc6\") pod \"migrator-66df5d496b-pdqh2\" (UID: \"662545a9-fe29-4583-8fdd-8ce384ee6681\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" Oct 30 03:04:09.742766 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.742732 2585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d6fwr"] Oct 30 03:04:09.764735 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.764655 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.767493 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.767463 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d6fwr"] Oct 30 03:04:09.780947 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.780924 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 03:04:09.806680 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.806650 2585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-wbbgs-f2qnw"] Oct 30 03:04:09.830376 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.830345 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-wbbgs-f2qnw"] Oct 30 03:04:09.830532 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.830474 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:09.840895 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.840877 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" Oct 30 03:04:09.879837 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.879809 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 03:04:09.879950 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.879856 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2x78k\"" Oct 30 03:04:09.879950 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.879866 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 03:04:09.879950 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.879878 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 03:04:09.879950 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.879882 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 03:04:09.880131 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.880121 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rwqxl\"" Oct 30 03:04:09.898591 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.898558 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-data-volume\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.898990 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.898614 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx9m2\" (UniqueName: \"kubernetes.io/projected/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-api-access-dx9m2\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.898990 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.898674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.898990 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.898712 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.898990 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.898790 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-crio-socket\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.899442 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.899416 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 03:04:09.999502 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999472 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-crio-socket\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.999682 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-data-volume\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.999682 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999570 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx9m2\" (UniqueName: \"kubernetes.io/projected/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-api-access-dx9m2\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.999682 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999605 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:09.999682 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999636 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.999682 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999659 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:09.999912 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999685 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:09.999912 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999700 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:09.999912 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999717 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xntc\" (UniqueName: \"kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.000035 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:09.999926 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-crio-socket\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:10.000272 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.000238 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-data-volume\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:10.000404 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.000318 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 03:04:10.000404 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.000344 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 03:04:10.000404 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.000388 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:10.500368288 +0000 UTC m=+34.382262491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : configmap "kube-rbac-proxy" not found Oct 30 03:04:10.000404 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.000406 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:10.500397201 +0000 UTC m=+34.382291372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : secret "insights-runtime-extractor-tls" not found Oct 30 03:04:10.027580 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.027510 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx9m2\" (UniqueName: \"kubernetes.io/projected/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-api-access-dx9m2\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:10.100292 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.100266 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.100292 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.100302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.100543 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.100334 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xntc\" (UniqueName: \"kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.100543 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.100411 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.100867 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.100848 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.101078 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.101058 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.101385 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.101363 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.133536 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.133497 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xntc\" (UniqueName: \"kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.153254 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.153229 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2"] Oct 30 03:04:10.154501 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:04:10.154477 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod662545a9_fe29_4583_8fdd_8ce384ee6681.slice/crio-a949307fd59b4a3af8606d75b7d61c11fc8a8edb0b2f759524455f705eb2b00b WatchSource:0}: Error finding container a949307fd59b4a3af8606d75b7d61c11fc8a8edb0b2f759524455f705eb2b00b: Status 404 returned error can't find the container with id a949307fd59b4a3af8606d75b7d61c11fc8a8edb0b2f759524455f705eb2b00b Oct 30 03:04:10.159640 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.159623 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:10.402375 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.402346 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:10.402538 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.402386 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:10.402538 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.402530 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 03:04:10.402620 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.402580 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs podName:59d544f8-cac5-4693-b82e-13f2ca910bfe nodeName:}" failed. No retries permitted until 2026-10-30 03:04:42.4025689 +0000 UTC m=+66.284463083 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs") pod "network-metrics-daemon-ks76k" (UID: "59d544f8-cac5-4693-b82e-13f2ca910bfe") : secret "metrics-daemon-secret" not found Oct 30 03:04:10.404555 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.404536 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:10.409446 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.409419 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:10.448091 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.448073 2585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-mg496"] Oct 30 03:04:10.469255 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.469232 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.471276 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.471255 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-mg496"] Oct 30 03:04:10.473689 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.473651 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-wbbgs-f2qnw"] Oct 30 03:04:10.475584 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:04:10.475541 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3608b350_8e02_4fc1_b866_3ef44f8ab2a9.slice/crio-566c550e6b2d4a9f3a6af414c165a450d69d85d32cdbed36ff344c545139a4c7 WatchSource:0}: Error finding container 566c550e6b2d4a9f3a6af414c165a450d69d85d32cdbed36ff344c545139a4c7: Status 404 returned error can't find the container with id 566c550e6b2d4a9f3a6af414c165a450d69d85d32cdbed36ff344c545139a4c7 Oct 30 03:04:10.488975 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.488005 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 03:04:10.488975 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.488216 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 03:04:10.488975 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.488269 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 03:04:10.488975 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.488299 2585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-x2bfb\"" Oct 30 03:04:10.488975 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.488756 2585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 03:04:10.503753 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.503729 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:10.503821 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.503781 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:10.503882 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.503843 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:10.503882 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.503865 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 03:04:10.503973 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.503932 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:11.50391085 +0000 UTC m=+35.385805033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : configmap "kube-rbac-proxy" not found Oct 30 03:04:10.503973 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.503935 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 03:04:10.504058 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.503989 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:11.503972655 +0000 UTC m=+35.385866839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : secret "insights-runtime-extractor-tls" not found Oct 30 03:04:10.506658 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.506640 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:10.604430 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.604400 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s42qh\" (UniqueName: \"kubernetes.io/projected/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-kube-api-access-s42qh\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.604540 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.604436 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-cabundle\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.604579 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.604537 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-key\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.703580 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.703528 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:10.706413 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.706170 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s42qh\" (UniqueName: \"kubernetes.io/projected/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-kube-api-access-s42qh\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.706413 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.706218 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-cabundle\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.706413 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.706278 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-key\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.709919 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.709895 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-key\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.710003 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.709965 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-cabundle\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.730055 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.730037 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2p9zt"] Oct 30 03:04:10.731498 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:04:10.731474 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bac3d0d_dded_4047_81d5_333e47612f93.slice/crio-9248a18ec011c2b7924a32dc5ad513bdf0e03ec64cc55bb1d21c8b97cdded04d WatchSource:0}: Error finding container 9248a18ec011c2b7924a32dc5ad513bdf0e03ec64cc55bb1d21c8b97cdded04d: Status 404 returned error can't find the container with id 9248a18ec011c2b7924a32dc5ad513bdf0e03ec64cc55bb1d21c8b97cdded04d Oct 30 03:04:10.736674 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.736653 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s42qh\" (UniqueName: \"kubernetes.io/projected/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-kube-api-access-s42qh\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.801268 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.801232 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:10.908076 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.908051 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:10.908355 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.908180 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 03:04:10.908355 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:10.908227 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls podName:d5170510-91aa-451e-bc7d-e18c3aaca68b nodeName:}" failed. No retries permitted until 2026-10-30 03:04:14.908215715 +0000 UTC m=+38.790109883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls") pod "dns-default-nxn6c" (UID: "d5170510-91aa-451e-bc7d-e18c3aaca68b") : secret "dns-default-metrics-tls" not found Oct 30 03:04:10.951863 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.951762 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" event={"ID":"3608b350-8e02-4fc1-b866-3ef44f8ab2a9","Type":"ContainerStarted","Data":"566c550e6b2d4a9f3a6af414c165a450d69d85d32cdbed36ff344c545139a4c7"} Oct 30 03:04:10.953313 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.953279 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" event={"ID":"662545a9-fe29-4583-8fdd-8ce384ee6681","Type":"ContainerStarted","Data":"a949307fd59b4a3af8606d75b7d61c11fc8a8edb0b2f759524455f705eb2b00b"} Oct 30 03:04:10.955087 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.955027 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b4nrx" event={"ID":"bdb6477c-e16b-451c-84bd-8e62d3efc79f","Type":"ContainerStarted","Data":"f498eeaca46d9f4412e5252ff3852522cce189ed7c4bab60b232edd2cad69f46"} Oct 30 03:04:10.956900 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:10.956880 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2p9zt" event={"ID":"1bac3d0d-dded-4047-81d5-333e47612f93","Type":"ContainerStarted","Data":"9248a18ec011c2b7924a32dc5ad513bdf0e03ec64cc55bb1d21c8b97cdded04d"} Oct 30 03:04:11.043541 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:11.042671 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b4nrx" podStartSLOduration=1.145446702 podStartE2EDuration="4.042650322s" totalImagesPullingTime="2.89720362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:11.000644506 +0000 UTC m=+34.882538696" watchObservedRunningTime="2026-10-30 03:04:11.042650322 +0000 UTC m=+34.924544515" Oct 30 03:04:11.044077 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:11.044010 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8cffw"] Oct 30 03:04:11.049172 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:04:11.049142 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe768a88_5dd3_44ff_800f_cd6603784ab4.slice/crio-145318213e86866f575c0db9ad9924dea6649432933c3195e6376c3902b48e20 WatchSource:0}: Error finding container 145318213e86866f575c0db9ad9924dea6649432933c3195e6376c3902b48e20: Status 404 returned error can't find the container with id 145318213e86866f575c0db9ad9924dea6649432933c3195e6376c3902b48e20 Oct 30 03:04:11.159000 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:11.158968 2585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-mg496"] Oct 30 03:04:11.160370 ip-10-0-132-171 kubenswrapper[2585]: W1030 03:04:11.160339 2585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb700fd45_b0e4_4cf2_a2d7_ed70c389bed8.slice/crio-e3d18ddc87b17807b0379547fc438e866a6c4c18e372684952a20eb40cbff475 WatchSource:0}: Error finding container e3d18ddc87b17807b0379547fc438e866a6c4c18e372684952a20eb40cbff475: Status 404 returned error can't find the container with id e3d18ddc87b17807b0379547fc438e866a6c4c18e372684952a20eb40cbff475 Oct 30 03:04:11.514406 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:11.513667 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:11.514406 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:11.513720 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:11.514406 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:11.513860 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 03:04:11.514406 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:11.513932 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:13.513914275 +0000 UTC m=+37.395808459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : secret "insights-runtime-extractor-tls" not found Oct 30 03:04:11.514406 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:11.514326 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 03:04:11.514406 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:11.514375 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:13.514360906 +0000 UTC m=+37.396255090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : configmap "kube-rbac-proxy" not found Oct 30 03:04:11.963193 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:11.963155 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" event={"ID":"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8","Type":"ContainerStarted","Data":"e3d18ddc87b17807b0379547fc438e866a6c4c18e372684952a20eb40cbff475"} Oct 30 03:04:11.965622 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:11.965507 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8cffw" event={"ID":"be768a88-5dd3-44ff-800f-cd6603784ab4","Type":"ContainerStarted","Data":"145318213e86866f575c0db9ad9924dea6649432933c3195e6376c3902b48e20"} Oct 30 03:04:12.971704 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:12.971664 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" event={"ID":"662545a9-fe29-4583-8fdd-8ce384ee6681","Type":"ContainerStarted","Data":"d408790d57537a157cc9fd1a41866fe75ba6bf1fc8cc503097b9509cd7c47637"} Oct 30 03:04:13.532111 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:13.532088 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:13.532223 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:13.532173 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:13.532288 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:13.532249 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 03:04:13.532288 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:13.532264 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 03:04:13.532379 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:13.532316 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:17.532299194 +0000 UTC m=+41.414193386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : configmap "kube-rbac-proxy" not found Oct 30 03:04:13.532379 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:13.532337 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:17.532326742 +0000 UTC m=+41.414220912 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : secret "insights-runtime-extractor-tls" not found Oct 30 03:04:13.978657 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:13.978590 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" event={"ID":"3608b350-8e02-4fc1-b866-3ef44f8ab2a9","Type":"ContainerStarted","Data":"857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8"} Oct 30 03:04:14.021843 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:14.021782 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" podStartSLOduration=3.176657652 podStartE2EDuration="5.021761689s" totalImagesPullingTime="1.845104037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:13.016327364 +0000 UTC m=+36.898221557" watchObservedRunningTime="2026-10-30 03:04:14.021761689 +0000 UTC m=+37.903655883" Oct 30 03:04:14.022374 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:14.022331 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" podStartSLOduration=1.9755497929999999 podStartE2EDuration="5.022321231s" totalImagesPullingTime="3.046771438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:14.021743915 +0000 UTC m=+37.903638104" watchObservedRunningTime="2026-10-30 03:04:14.022321231 +0000 UTC m=+37.904215423" Oct 30 03:04:14.943622 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:14.943540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:14.943817 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:14.943713 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 03:04:14.943817 ip-10-0-132-171 kubenswrapper[2585]: E1030 03:04:14.943791 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls podName:d5170510-91aa-451e-bc7d-e18c3aaca68b nodeName:}" failed. No retries permitted until 2026-10-30 03:04:22.943769326 +0000 UTC m=+46.825663500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls") pod "dns-default-nxn6c" (UID: "d5170510-91aa-451e-bc7d-e18c3aaca68b") : secret "dns-default-metrics-tls" not found Oct 30 03:04:14.981972 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:14.981938 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" event={"ID":"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8","Type":"ContainerStarted","Data":"6b3b9173abd3c8fcd9b67b7b9bdbb9bffd86ddeda973cbddd2c7aa369e48f57c"} Oct 30 03:04:15.025929 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:15.025884 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" podStartSLOduration=1.548084744 podStartE2EDuration="5.025872066s" totalImagesPullingTime="3.477787322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:15.025310871 +0000 UTC m=+38.907205070" watchObservedRunningTime="2026-10-30 03:04:15.025872066 +0000 UTC m=+38.907766300" Oct 30 03:04:15.987723 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:15.987687 2585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8cffw" event={"ID":"be768a88-5dd3-44ff-800f-cd6603784ab4","Type":"ContainerStarted","Data":"06fb5a64efc3f09efa38ada984a8e24e2db032bd8abbe566866d0a6acbc7bed8"} Oct 30 03:04:16.046258 ip-10-0-132-171 kubenswrapper[2585]: I1030 03:04:16.046205 2585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8cffw" podStartSLOduration=35.902543206 podStartE2EDuration="40.046189871s" totalImagesPullingTime="4.143646665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:16.045771497 +0000 UTC m=+39.927665688" watchObservedRunningTime="2026-10-30 03:04:16.046189871 +0000 UTC m=+39.928084067" Oct 30 03:04:16.794934 ip-10-0-132-171 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 03:04:16.806949 ip-10-0-132-171 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 03:04:16.807118 ip-10-0-132-171 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 03:04:16.807166 ip-10-0-132-171 systemd[1]: kubelet.service: Consumed 2.316s CPU time. Oct 30 03:04:16.826114 ip-10-0-132-171 systemd[1]: Starting Kubernetes Kubelet... Oct 30 03:04:16.937840 ip-10-0-132-171 kubenswrapper[4524]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 03:04:16.937840 ip-10-0-132-171 kubenswrapper[4524]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 03:04:16.937840 ip-10-0-132-171 kubenswrapper[4524]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 03:04:16.937840 ip-10-0-132-171 kubenswrapper[4524]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 03:04:16.939796 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939718 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:04:16.939796 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939794 4524 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939800 4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939805 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939808 4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939811 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939815 4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939817 4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939820 4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939823 4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939826 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939829 4524 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939832 4524 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939834 4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939837 4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939840 4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939842 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939845 4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939848 4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939851 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:04:16.939866 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939853 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939862 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939865 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939868 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939871 4524 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939874 4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939877 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939880 4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939882 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939885 4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939888 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939891 4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939893 4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939896 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939898 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939901 4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939904 4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939906 4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939909 4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:04:16.940339 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939912 4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939914 4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939917 4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939919 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939922 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939924 4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939927 4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939931 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939933 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939936 4524 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939939 4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939942 4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939944 4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939947 4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939952 4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939956 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939959 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939962 4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939966 4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:04:16.940841 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939969 4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939972 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939974 4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939977 4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939980 4524 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939983 4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939986 4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939988 4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939991 4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939994 4524 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939996 4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.939999 4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940002 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940005 4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940007 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940010 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940013 4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940016 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940019 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940022 4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:04:16.941330 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940024 4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940027 4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940030 4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940032 4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940035 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940038 4524 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940040 4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940043 4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940046 4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940049 4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940051 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940054 4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940057 4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940059 4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940062 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940064 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940067 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940070 4524 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940072 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940075 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:04:16.941885 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940078 4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940080 4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940083 4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940085 4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940088 4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940091 4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940093 4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940097 4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940837 4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940844 4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940848 4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940851 4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940854 4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940856 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940859 4524 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940862 4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940865 4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940868 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940871 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:04:16.942370 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940874 4524 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940877 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940879 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940882 4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940884 4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940887 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940890 4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940892 4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940896 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940899 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940901 4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940904 4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940907 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940909 4524 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940912 4524 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940915 4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940917 4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940920 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940923 4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940926 4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:04:16.942861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940929 4524 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940931 4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940934 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940937 4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940940 4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940943 4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940945 4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940948 4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940950 4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940953 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940955 4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940958 4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940961 4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940963 4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940966 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940968 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940971 4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940974 4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940976 4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940979 4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:04:16.943351 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940982 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940985 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940988 4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.940990 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941002 4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941005 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941008 4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941011 4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941013 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941016 4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941019 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941022 4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941025 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941027 4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941032 4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941036 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941040 4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941043 4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941045 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:04:16.943861 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941048 4524 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941051 4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941054 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941057 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941060 4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941063 4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941065 4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941068 4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941071 4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941075 4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941078 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941081 4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941084 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941087 4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941090 4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941093 4524 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941095 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941098 4524 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941101 4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:04:16.944323 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941103 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941106 4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941108 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941111 4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941114 4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941116 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941119 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941122 4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941124 4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941127 4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941130 4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941132 4524 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941135 4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941138 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941140 4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941143 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941147 4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941615 4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941620 4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941624 4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:04:16.944830 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941627 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941630 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941632 4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941635 4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941638 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941641 4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941644 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941646 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941649 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941651 4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941654 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941657 4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941659 4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941662 4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941665 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941668 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941670 4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941673 4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941675 4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941678 4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:04:16.945337 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941681 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941684 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941687 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941689 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941692 4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941695 4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941697 4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941700 4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941704 4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941708 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941714 4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941717 4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941720 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941723 4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941726 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941728 4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941732 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941734 4524 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941737 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:04:16.945927 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941740 4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941743 4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941745 4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941748 4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941751 4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941753 4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941756 4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941758 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941761 4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941764 4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941766 4524 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941770 4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941772 4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941775 4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941778 4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941780 4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941783 4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941786 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941788 4524 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941791 4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:04:16.946723 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941794 4524 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941796 4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941799 4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941803 4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941806 4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941808 4524 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941811 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941813 4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941816 4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941821 4524 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941824 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941826 4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941829 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941831 4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941834 4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941837 4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941840 4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941842 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941845 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941847 4524 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:04:16.947335 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941850 4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941853 4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941855 4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941858 4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941861 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941863 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941866 4524 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941869 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941872 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941874 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941877 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941879 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941882 4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941886 4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941889 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941895 4524 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941898 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941901 4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941904 4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:04:16.947848 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941907 4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941910 4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941914 4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941916 4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.941919 4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942006 4524 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942013 4524 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942019 4524 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942024 4524 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942029 4524 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942032 4524 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942036 4524 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942041 4524 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942044 4524 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942048 4524 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942051 4524 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942055 4524 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942058 4524 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942061 4524 flags.go:64] FLAG: --cgroup-root="" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942064 4524 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942067 4524 flags.go:64] FLAG: --client-ca-file="" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942071 4524 flags.go:64] FLAG: --cloud-provider="external" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942075 4524 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 03:04:16.948300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942081 4524 flags.go:64] FLAG: --cluster-domain="" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942086 4524 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942089 4524 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942092 4524 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942096 4524 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942102 4524 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942105 4524 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942108 4524 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942111 4524 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942115 4524 flags.go:64] FLAG: --contention-profiling="false" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942118 4524 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942122 4524 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942125 4524 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942128 4524 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942133 4524 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942136 4524 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942139 4524 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942142 4524 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942145 4524 flags.go:64] FLAG: --enable-server="true" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942148 4524 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942153 4524 flags.go:64] FLAG: --event-burst="100" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942156 4524 flags.go:64] FLAG: --event-qps="50" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942159 4524 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942162 4524 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 03:04:16.948970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942165 4524 flags.go:64] FLAG: --eviction-hard="" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942169 4524 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942172 4524 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942175 4524 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942178 4524 flags.go:64] FLAG: --eviction-soft="" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942182 4524 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942185 4524 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942189 4524 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942192 4524 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942195 4524 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942198 4524 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942201 4524 flags.go:64] FLAG: --feature-gates="" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942205 4524 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942209 4524 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942212 4524 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942216 4524 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942218 4524 flags.go:64] FLAG: --healthz-port="10248" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942222 4524 flags.go:64] FLAG: --help="false" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942225 4524 flags.go:64] FLAG: --hostname-override="ip-10-0-132-171.ec2.internal" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942229 4524 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942232 4524 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942235 4524 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942239 4524 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 03:04:16.949637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942242 4524 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942245 4524 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942248 4524 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942251 4524 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942254 4524 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942257 4524 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942260 4524 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942263 4524 flags.go:64] FLAG: --kube-reserved="" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942266 4524 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942269 4524 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942273 4524 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942276 4524 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942279 4524 flags.go:64] FLAG: --lock-file="" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942282 4524 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942285 4524 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942288 4524 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942294 4524 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942297 4524 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942300 4524 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942303 4524 flags.go:64] FLAG: --logging-format="text" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942306 4524 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942310 4524 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942314 4524 flags.go:64] FLAG: --manifest-url="" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942317 4524 flags.go:64] FLAG: --manifest-url-header="" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942321 4524 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 03:04:16.950209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942325 4524 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942329 4524 flags.go:64] FLAG: --max-pods="110" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942332 4524 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942337 4524 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942340 4524 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942343 4524 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942346 4524 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942349 4524 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942352 4524 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942359 4524 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942362 4524 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942365 4524 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942368 4524 flags.go:64] FLAG: --pod-cidr="" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942372 4524 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942375 4524 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942379 4524 flags.go:64] FLAG: --pods-per-core="0" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942382 4524 flags.go:64] FLAG: --port="10250" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942386 4524 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942389 4524 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fff32d287d94d358" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942392 4524 flags.go:64] FLAG: --qos-reserved="" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942396 4524 flags.go:64] FLAG: --read-only-port="10255" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942399 4524 flags.go:64] FLAG: --register-node="true" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942402 4524 flags.go:64] FLAG: --register-with-taints="" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942406 4524 flags.go:64] FLAG: --registry-burst="10" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942409 4524 flags.go:64] FLAG: --registry-qps="5" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942412 4524 flags.go:64] FLAG: --reserved-cpus="" Oct 30 03:04:16.950832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942416 4524 flags.go:64] FLAG: --reserved-memory="" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942419 4524 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942423 4524 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942426 4524 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942429 4524 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942432 4524 flags.go:64] FLAG: --runonce="false" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942436 4524 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942439 4524 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942442 4524 flags.go:64] FLAG: --seccomp-default="false" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942450 4524 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942453 4524 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942456 4524 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942459 4524 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942462 4524 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942465 4524 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942468 4524 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942471 4524 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942474 4524 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942478 4524 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942481 4524 flags.go:64] FLAG: --system-cgroups="" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942484 4524 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942489 4524 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942492 4524 flags.go:64] FLAG: --tls-cert-file="" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942495 4524 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942500 4524 flags.go:64] FLAG: --tls-min-version="" Oct 30 03:04:16.951511 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942503 4524 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942506 4524 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942509 4524 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942526 4524 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942529 4524 flags.go:64] FLAG: --v="2" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942534 4524 flags.go:64] FLAG: --version="false" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942538 4524 flags.go:64] FLAG: --vmodule="" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942542 4524 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942546 4524 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942670 4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942674 4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942678 4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942682 4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942685 4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942689 4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942693 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942698 4524 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942701 4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942704 4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942707 4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942709 4524 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:04:16.952140 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942712 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942715 4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942718 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942721 4524 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942724 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942727 4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942729 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942732 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942735 4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942737 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942740 4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942743 4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942746 4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942748 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942751 4524 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942754 4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942757 4524 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942760 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942763 4524 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942766 4524 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:04:16.952682 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942768 4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942771 4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942774 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942777 4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942779 4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942782 4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942785 4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942789 4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942791 4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942794 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942797 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942799 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942802 4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942805 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942808 4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942811 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942813 4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942816 4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942818 4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:04:16.953182 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942821 4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942824 4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942826 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942829 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942832 4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942834 4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942837 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942840 4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942842 4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942845 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942848 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942850 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942853 4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942856 4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942858 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942862 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942864 4524 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942867 4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942870 4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:04:16.953817 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942873 4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942877 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942880 4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942882 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942885 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942888 4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942890 4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942893 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942896 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942898 4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942901 4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942903 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942906 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942910 4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942914 4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942916 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942919 4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942922 4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942924 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:04:16.954345 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942927 4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942930 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942933 4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942936 4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942938 4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942941 4524 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942943 4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942946 4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942949 4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942952 4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942954 4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942957 4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942959 4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942962 4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942966 4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942969 4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:04:16.955188 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.942972 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.942996 4524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943031 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943035 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943039 4524 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943042 4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943046 4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943049 4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943052 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943054 4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943057 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943061 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943064 4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943067 4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943069 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943072 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:04:16.955931 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943074 4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943077 4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943080 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943083 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943086 4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943088 4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943091 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943094 4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943097 4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943100 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943103 4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943106 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943109 4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943111 4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943115 4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943118 4524 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943121 4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943124 4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943126 4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943129 4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:04:16.956423 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943132 4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943134 4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943137 4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943140 4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943142 4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943145 4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943148 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943152 4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943154 4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943158 4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943162 4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943164 4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943169 4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943172 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943175 4524 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943178 4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943180 4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943183 4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943186 4524 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:04:16.957067 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943189 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943191 4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943194 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943197 4524 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943199 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943202 4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943205 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943208 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943211 4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943214 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943217 4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943219 4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943222 4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943225 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943228 4524 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943231 4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943233 4524 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943236 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943239 4524 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943241 4524 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:04:16.957736 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943245 4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943248 4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943250 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943253 4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943256 4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943258 4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943261 4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943264 4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943266 4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943269 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943272 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943274 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943277 4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943281 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943283 4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943286 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943289 4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943291 4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943294 4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:04:16.958241 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943298 4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943301 4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943304 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943306 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943309 4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943311 4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943315 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943317 4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943320 4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943323 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943325 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943328 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943331 4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.943335 4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.948417 4524 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.948434 4524 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.948458 4524 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.948463 4524 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948534 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:04:16.958720 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948540 4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948544 4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948547 4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948550 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948553 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948556 4524 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948559 4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948562 4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948565 4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948569 4524 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948572 4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948574 4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948577 4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948580 4524 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948583 4524 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948585 4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948588 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948590 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948593 4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948596 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:04:16.959180 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948599 4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948615 4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948620 4524 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948623 4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948627 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948630 4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948633 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948640 4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948643 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948646 4524 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948648 4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948651 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948654 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948657 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948660 4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948662 4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948665 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948667 4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948670 4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:04:16.959687 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948673 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948676 4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948679 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948682 4524 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948685 4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948687 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948690 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948693 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948695 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948698 4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948701 4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948703 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948706 4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948709 4524 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948712 4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948726 4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948729 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948732 4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948744 4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948748 4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:04:16.960137 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948751 4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948755 4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948758 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948761 4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948764 4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948767 4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948770 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948773 4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948775 4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948779 4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948783 4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948786 4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948788 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948791 4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948794 4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948797 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948799 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948802 4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948804 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:04:16.960651 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948808 4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948811 4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948813 4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948816 4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948819 4524 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948822 4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948824 4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948827 4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948830 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948832 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948835 4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948837 4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948840 4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948843 4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948845 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948848 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948850 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948853 4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948856 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948858 4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:04:16.961184 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948861 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948864 4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948867 4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948869 4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948872 4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948875 4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.948878 4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.948906 4524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949041 4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949045 4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949048 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949051 4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949054 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949057 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949060 4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949063 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:04:16.962060 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949066 4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949068 4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949071 4524 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949074 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949077 4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949079 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949082 4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949085 4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949088 4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949090 4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949093 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949095 4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949098 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949100 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949103 4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949106 4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949109 4524 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949111 4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949114 4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949117 4524 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:04:16.962543 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949119 4524 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949122 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949125 4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949128 4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949130 4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949133 4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949136 4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949138 4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949141 4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949144 4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949147 4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949150 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949152 4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949155 4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949158 4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949160 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949163 4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949165 4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949168 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:04:16.963017 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949170 4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949173 4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949176 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949178 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949181 4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949184 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949186 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949189 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949193 4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949196 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949198 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949201 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949204 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949206 4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949210 4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949214 4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949218 4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949220 4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:04:16.963489 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949223 4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949226 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949229 4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949231 4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949234 4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949236 4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949240 4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949243 4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949245 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949249 4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949253 4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949256 4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949259 4524 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949261 4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949264 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949267 4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949270 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949272 4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949275 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949278 4524 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:04:16.963945 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949280 4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949283 4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949286 4524 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949289 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949291 4524 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949294 4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949296 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949299 4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949302 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949304 4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949307 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949309 4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949312 4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949315 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949317 4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949320 4524 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949322 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949325 4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949328 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949331 4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:04:16.964462 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949334 4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.949358 4524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949390 4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949395 4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949398 4524 feature_gate.go:386] unrecognized feature gate: Example Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949400 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949403 4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949406 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949408 4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949411 4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949414 4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949416 4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949419 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949421 4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949424 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949427 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 03:04:16.964979 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949429 4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949432 4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949435 4524 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949437 4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949440 4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949443 4524 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949445 4524 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949448 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949451 4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949453 4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949456 4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949458 4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949461 4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949464 4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949466 4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949469 4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949472 4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949475 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949478 4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949480 4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 03:04:16.965386 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949483 4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949486 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949488 4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949491 4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949493 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949496 4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949499 4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949501 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949526 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949530 4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949532 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949535 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949538 4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949541 4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949543 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949546 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949549 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949551 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949554 4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 03:04:16.965995 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949558 4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949563 4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949566 4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949569 4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949575 4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949578 4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949581 4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949583 4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949586 4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949589 4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949592 4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949595 4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949598 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949602 4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949605 4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949608 4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949611 4524 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949614 4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949617 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 03:04:16.966467 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949619 4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949622 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949625 4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949627 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949630 4524 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949634 4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949637 4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949640 4524 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949643 4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949646 4524 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949648 4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949651 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949654 4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949657 4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949660 4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949663 4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949665 4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949670 4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949672 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949675 4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 03:04:16.966938 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949678 4524 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949680 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949683 4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949686 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949689 4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949692 4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949695 4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949697 4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949700 4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949703 4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949705 4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949708 4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949712 4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:16.949714 4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.949771 4524 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.950746 4524 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.950802 4524 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.950984 4524 server.go:1014] "Starting client certificate rotation" Oct 30 03:04:16.967641 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.951118 4524 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 03:04:16.968486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.951171 4524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 02:58:36 +0000 UTC" deadline="2028-06-28 20:09:09.01087489 +0000 UTC" Oct 30 03:04:16.968486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.951199 4524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14585h4m52.059677294s" Oct 30 03:04:16.968486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.951457 4524 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 03:04:16.968486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.952504 4524 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 03:04:16.968486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.954057 4524 log.go:25] "Validated CRI v1 runtime API" Oct 30 03:04:16.968486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.956080 4524 log.go:25] "Validated CRI v1 image API" Oct 30 03:04:16.968486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.956877 4524 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 03:04:16.968486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.958409 4524 fs.go:135] Filesystem UUIDs: map[09df95af-1b30-4872-915d-e67521ed2102:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 dcd426c2-9e85-4515-9106-9696fa7f0ea4:/dev/nvme0n1p3] Oct 30 03:04:16.968887 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.958428 4524 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/143ae3ac6af31cbca158f8457cd5debdc7ef2a07b7c508d9dcd71ef1378691bf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/143ae3ac6af31cbca158f8457cd5debdc7ef2a07b7c508d9dcd71ef1378691bf/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/145318213e86866f575c0db9ad9924dea6649432933c3195e6376c3902b48e20/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/145318213e86866f575c0db9ad9924dea6649432933c3195e6376c3902b48e20/userdata/shm major:0 minor:225 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/14ef22c78b45926184f1ff02ee96b2def3755aaea1a9e1114f1c7e284fa44d1a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/14ef22c78b45926184f1ff02ee96b2def3755aaea1a9e1114f1c7e284fa44d1a/userdata/shm major:0 minor:69 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/168dd325ea4c992d8739825170c9cabf4d9d930c68c308b377d787bc1b159b57/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/168dd325ea4c992d8739825170c9cabf4d9d930c68c308b377d787bc1b159b57/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/566c550e6b2d4a9f3a6af414c165a450d69d85d32cdbed36ff344c545139a4c7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/566c550e6b2d4a9f3a6af414c165a450d69d85d32cdbed36ff344c545139a4c7/userdata/shm major:0 minor:213 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/56a70dd10da311078ba4997ed73aeda78169deebedf9de4447bccda37f4d2d5a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/56a70dd10da311078ba4997ed73aeda78169deebedf9de4447bccda37f4d2d5a/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5b7cfbc5319e7f442eea768b44a14b523b9e5edfd84298635a40d58ca3b39671/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5b7cfbc5319e7f442eea768b44a14b523b9e5edfd84298635a40d58ca3b39671/userdata/shm major:0 minor:87 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74617c2f962a27e1da516a2cc4d931a206d42110349bc68969a6e6402fcaaaaa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74617c2f962a27e1da516a2cc4d931a206d42110349bc68969a6e6402fcaaaaa/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9248a18ec011c2b7924a32dc5ad513bdf0e03ec64cc55bb1d21c8b97cdded04d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9248a18ec011c2b7924a32dc5ad513bdf0e03ec64cc55bb1d21c8b97cdded04d/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c8f3d51d1799a83759828681971124ce841d8a032e9f04d9497febb5277a8ff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c8f3d51d1799a83759828681971124ce841d8a032e9f04d9497febb5277a8ff/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a949307fd59b4a3af8606d75b7d61c11fc8a8edb0b2f759524455f705eb2b00b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a949307fd59b4a3af8606d75b7d61c11fc8a8edb0b2f759524455f705eb2b00b/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e3d18ddc87b17807b0379547fc438e866a6c4c18e372684952a20eb40cbff475/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e3d18ddc87b17807b0379547fc438e866a6c4c18e372684952a20eb40cbff475/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e4e7ca17a2a73713d132705c2942e4e37f78afab4691539f23103ded1c329067/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e4e7ca17a2a73713d132705c2942e4e37f78afab4691539f23103ded1c329067/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e694b13bb4c8ba60b7ab3c256513262d4a8cc5d4cc547880d9254dd9a9094d3d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e694b13bb4c8ba60b7ab3c256513262d4a8cc5d4cc547880d9254dd9a9094d3d/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eccd84d9e3c2a2ab6e44722edbc11c8f09d729fcfcaf3d9dc7e3756ac70a2943/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eccd84d9e3c2a2ab6e44722edbc11c8f09d729fcfcaf3d9dc7e3756ac70a2943/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5c361011544f41d06155e7697e42b2f1385e67addaff7ba4584c0ae6bff14f5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5c361011544f41d06155e7697e42b2f1385e67addaff7ba4584c0ae6bff14f5/userdata/shm major:0 minor:99 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bb13fe7-8a37-41d7-90aa-fed271998088/volumes/kubernetes.io~projected/kube-api-access-jn5xx:{mountpoint:/var/lib/kubelet/pods/0bb13fe7-8a37-41d7-90aa-fed271998088/volumes/kubernetes.io~projected/kube-api-access-jn5xx major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1accce46-e302-4151-824a-5d9232a1be2b/volumes/kubernetes.io~projected/kube-api-access-jg72b:{mountpoint:/var/lib/kubelet/pods/1accce46-e302-4151-824a-5d9232a1be2b/volumes/kubernetes.io~projected/kube-api-access-jg72b major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1bac3d0d-dded-4047-81d5-333e47612f93/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/1bac3d0d-dded-4047-81d5-333e47612f93/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45/volumes/kubernetes.io~projected/kube-api-access-6hm8w:{mountpoint:/var/lib/kubelet/pods/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45/volumes/kubernetes.io~projected/kube-api-access-6hm8w major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/31deab60-7ba7-431c-9fdd-f15388845315/volumes/kubernetes.io~projected/kube-api-access-6nhkc:{mountpoint:/var/lib/kubelet/pods/31deab60-7ba7-431c-9fdd-f15388845315/volumes/kubernetes.io~projected/kube-api-access-6nhkc major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3608b350-8e02-4fc1-b866-3ef44f8ab2a9/volumes/kubernetes.io~projected/kube-api-access-9xntc:{mountpoint:/var/lib/kubelet/pods/3608b350-8e02-4fc1-b866-3ef44f8ab2a9/volumes/kubernetes.io~projected/kube-api-access-9xntc major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f/volumes/kubernetes.io~projected/kube-api-access-dx9m2:{mountpoint:/var/lib/kubelet/pods/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f/volumes/kubernetes.io~projected/kube-api-access-dx9m2 major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/59d544f8-cac5-4693-b82e-13f2ca910bfe/volumes/kubernetes.io~projected/kube-api-access-sgtzl:{mountpoint:/var/lib/kubelet/pods/59d544f8-cac5-4693-b82e-13f2ca910bfe/volumes/kubernetes.io~projected/kube-api-access-sgtzl major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/662545a9-fe29-4583-8fdd-8ce384ee6681/volumes/kubernetes.io~projected/kube-api-access-2zrc6:{mountpoint:/var/lib/kubelet/pods/662545a9-fe29-4583-8fdd-8ce384ee6681/volumes/kubernetes.io~projected/kube-api-access-2zrc6 major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volumes/kubernetes.io~projected/kube-api-access-6g62l:{mountpoint:/var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volumes/kubernetes.io~projected/kube-api-access-6g62l major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~empty-dir/tmp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~projected/kube-api-access-wptw5:{mountpoint:/var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~projected/kube-api-access-wptw5 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9a093097-53a0-40a5-9152-0e2930424cec/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/9a093097-53a0-40a5-9152-0e2930424cec/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8/volumes/kubernetes.io~projected/kube-api-access-s42qh:{mountpoint:/var/lib/kubelet/pods/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8/volumes/kubernetes.io~projected/kube-api-access-s42qh major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8/volumes/kubernetes.io~secret/signing-key major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdb6477c-e16b-451c-84bd-8e62d3efc79f/volumes/kubernetes.io~projected/kube-api-access-xlj67:{mountpoint:/var/lib/kubelet/pods/bdb6477c-e16b-451c-84bd-8e62d3efc79f/volumes/kubernetes.io~projected/kube-api-access-xlj67 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdb6477c-e16b-451c-84bd-8e62d3efc79f/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/bdb6477c-e16b-451c-84bd-8e62d3efc79f/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/be768a88-5dd3-44ff-800f-cd6603784ab4/volumes/kubernetes.io~projected/kube-api-access-6bcnm:{mountpoint:/var/lib/kubelet/pods/be768a88-5dd3-44ff-800f-cd6603784ab4/volumes/kubernetes.io~projected/kube-api-access-6bcnm major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d5170510-91aa-451e-bc7d-e18c3aaca68b/volumes/kubernetes.io~projected/kube-api-access-rlprd:{mountpoint:/var/lib/kubelet/pods/d5170510-91aa-451e-bc7d-e18c3aaca68b/volumes/kubernetes.io~projected/kube-api-access-rlprd major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/da1b6928-2507-4293-8e3c-f548c5b8ffab/volumes/kubernetes.io~projected/kube-api-access-nh86z:{mountpoint:/var/lib/kubelet/pods/da1b6928-2507-4293-8e3c-f548c5b8ffab/volumes/kubernetes.io~projected/kube-api-access-nh86z major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1078aa9-08f9-482d-b68c-16163edfaad8/volumes/kubernetes.io~projected/kube-api-access-wlncr:{mountpoint:/var/lib/kubelet/pods/f1078aa9-08f9-482d-b68c-16163edfaad8/volumes/kubernetes.io~projected/kube-api-access-wlncr major:0 minor:59 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/a6db607887a74dd3e2cc0b112573389e3a692613619045100489c768664d094a/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/671fd97a5c31e5557c2cff8cfb4e2829f8ef669bb60c6ac17fdd5508792cbb9f/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/013d595be21995df2fd3eed18f7540e29f299ef483499099f5b2190cf81eab0f/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/c6808b5084998831932908a2f65df7a591b9c653bfc35e996142a1c1766b896d/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/9ad3a31c9429727b2fa4422e4f1efe2632fd10dde0de5a57163fe46a1f5b77e3/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/f6fc9226e8f093c4747d6118e0ba63d4255303289fe47ebe0616599cc558dbf9/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/2dae914fc6c4b00c6c7326969d15038dc53e2f850d947c70b07c27edc419755c/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/2dfca95a1b90a86f8aebcc6cfa6d9a7052d1a0f5a23d64be3ffb8578e89e865a/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/36f293a5bf1faf073b650e270767b22d1894f46ff79ca791b168d0e5e5d85624/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/c9763567cb093d7c567cb4ba235cecfc14597c01fea4a0fab1c6529035279b09/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-145:{mountpoint:/var/lib/containers/storage/overlay/3f4bf01172785d1a4d87c7e3f39a454aa01e9c02d37148ec0b9bb949bde47002/merged major:0 minor:145 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/860106dc0515fdc7ebb39c409b1f1e191a6c6f737516103004c6902607b45c5c/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/629c23b8b2c6082e4e86fdc6b3e2e776756692d9c97c63f56c780c77fc30fc66/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-155:{mountpoint:/var/lib/containers/storage/overlay/6ee39528ca9a2e90e76ba36efdee3e2d753ad18f998f77f5d1ad293997b7d1f1/merged major:0 minor:155 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/07514388fe77518befd73d1734800aa2a310dc68f62cb27436099ebd774b0c7d/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/428b3eaa0a393179e81461b3019d3140d375c5dac6cf44ad27c0d3d35f6b34a3/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/1b28cc6c3c514d6f31544e315697f099e5e0e5091e7565bb962316067885a710/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/f6aae7c8c9af76010fe57a490302d15e728260ab790c19da62bbc19862c1487b/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/d6fcb173bdd3b437221c495b3793ec4d68523b054666960e6408fa8b12a1a3ff/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/be8d0cc2454cf94b63ce61438bcaafbc933dde78a92947a81fcf5323fa43ccca/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/68f82fc3fcb80108f4ca420633f146308576280de0d6922db6a6d931873b0c8f/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/38dadadfdeb92ef66b085e2cd913fa74ed6c971029a8ced3eb6f548d1025b1c5/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-218:{mountpoint:/var/lib/containers/storage/overlay/81519993e48073c0ef7735b95819e6225e42a217a6c26a0a88f45e6c75dbc66f/merged major:0 minor:218 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/8c571aada60d6d37ae32f2a58ce19ae174e8ab647a704daba5e2bb89d34c03b1/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/258d9f6eb115068e9907f4899d358611d17f48ce468e0bffe16d228178ad82aa/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-229:{mountpoint:/var/lib/containers/storage/overlay/1d93a414feda78e05ffa5b98a34707f51b7e71cbebe2459529e640a6160e56c8/merged major:0 minor:229 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/cd3d12b06c2b374232b66c16e4c07da8de5014f9bfb909f346d1a7b9a469559d/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/c170ca2972cd439c7ede8f108b5760793abfea07d3f7eed1cbb866c5a14bef44/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/a256e4270ad6450aa355514b6881b70c2c2de2a604440acd0ee2ecc420f1a735/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/84708881dbfe347005557c9a2f1f81c0100a1380d44b8790cc99508c082c0d05/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/e65a1c91b7eb10e2fcfb8109a7baf09686eaa946a7782d4b358a98ec909146a3/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/02ab7782ac9f3389cba23f1fee32c4b8fd570a2c4963b321b8827c3e00dda780/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/c148ae40906c330f1eae51b6c3f5d5f4d34798c72eb6e0935b637487522d4a1e/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/5b8ceb3a907a654ba38496bd0e66fb8f5c7ca59f267cb970a2406bc9c982400e/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/1891d95f1ef398504ed7a4a34fa706b020c8b51042f173f7b6bf91ebedb1a79a/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/72827b290ca54e6768f15b35210890f6f0f3421a30ce2eb0500f06b5239c3c8b/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/03fb939555e8a51fce7ff2f7e4ebfafc2268f6313b8cc50ffaf3531fcf3d97f5/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/44d7d6462f634359650e829eca18ccf7125fe09f8c418e89ddf56ed260f6ab3b/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/e9464cadb365b47fef9807d623a9a6f1ce72f5dd2a5dfb6764018c3520c700ab/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/52cfef3dd4d23ebe1877fc7b62414e4c66fd9a5d63056818fcec87b8614a8bef/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/bdc5613811eb5f7d3bd44fd905797e494b3f21bf2df49c338ac3d6fa874a5541/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/7a417e52b41b155cdd61adaed55d5cf4a51d3cf1840a04be5a3927b0724f560b/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/f54b66bce1ce7a5abff39315f9dbdd53d4a8bd7b1935edb9cb7964f42df7ec21/merged major:0 minor:98 fsType:overlay blockSize:0}] Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.965944 4524 manager.go:217] Machine: {Timestamp:2026-10-30 03:04:16.965480255 +0000 UTC m=+0.087046814 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097300 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d8a0932c64a36ded936fc52f9c335 SystemUUID:ec2d8a09-32c6-4a36-ded9-36fc52f9c335 BootID:88063c7f-b94f-4f9a-aeec-4a9863035b1b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e694b13bb4c8ba60b7ab3c256513262d4a8cc5d4cc547880d9254dd9a9094d3d/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74617c2f962a27e1da516a2cc4d931a206d42110349bc68969a6e6402fcaaaaa/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f/volumes/kubernetes.io~projected/kube-api-access-dx9m2 DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-155 DeviceMajor:0 DeviceMinor:155 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3608b350-8e02-4fc1-b866-3ef44f8ab2a9/volumes/kubernetes.io~projected/kube-api-access-9xntc DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-218 DeviceMajor:0 DeviceMinor:218 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bdb6477c-e16b-451c-84bd-8e62d3efc79f/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9248a18ec011c2b7924a32dc5ad513bdf0e03ec64cc55bb1d21c8b97cdded04d/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/145318213e86866f575c0db9ad9924dea6649432933c3195e6376c3902b48e20/userdata/shm DeviceMajor:0 DeviceMinor:225 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-145 DeviceMajor:0 DeviceMinor:145 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/31deab60-7ba7-431c-9fdd-f15388845315/volumes/kubernetes.io~projected/kube-api-access-6nhkc DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f5c361011544f41d06155e7697e42b2f1385e67addaff7ba4584c0ae6bff14f5/userdata/shm DeviceMajor:0 DeviceMinor:99 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:221 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e3d18ddc87b17807b0379547fc438e866a6c4c18e372684952a20eb40cbff475/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/143ae3ac6af31cbca158f8457cd5debdc7ef2a07b7c508d9dcd71ef1378691bf/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/662545a9-fe29-4583-8fdd-8ce384ee6681/volumes/kubernetes.io~projected/kube-api-access-2zrc6 DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~projected/kube-api-access-wptw5 DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volumes/kubernetes.io~projected/kube-api-access-6g62l DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/f1078aa9-08f9-482d-b68c-16163edfaad8/volumes/kubernetes.io~projected/kube-api-access-wlncr DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/da1b6928-2507-4293-8e3c-f548c5b8ffab/volumes/kubernetes.io~projected/kube-api-access-nh86z DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/566c550e6b2d4a9f3a6af414c165a450d69d85d32cdbed36ff344c545139a4c7/userdata/shm DeviceMajor:0 DeviceMinor:213 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9a093097-53a0-40a5-9152-0e2930424cec/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a949307fd59b4a3af8606d75b7d61c11fc8a8edb0b2f759524455f705eb2b00b/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1bac3d0d-dded-4047-81d5-333e47612f93/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:215 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/be768a88-5dd3-44ff-800f-cd6603784ab4/volumes/kubernetes.io~projected/kube-api-access-6bcnm DeviceMajor:0 DeviceMinor:220 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-229 DeviceMajor:0 DeviceMinor:229 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45/volumes/kubernetes.io~projected/kube-api-access-6hm8w DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bdb6477c-e16b-451c-84bd-8e62d3efc79f/volumes/kubernetes.io~projected/kube-api-access-xlj67 DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eccd84d9e3c2a2ab6e44722edbc11c8f09d729fcfcaf3d9dc7e3756ac70a2943/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8/volumes/kubernetes.io~projected/kube-api-access-s42qh DeviceMajor:0 DeviceMinor:224 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/983ff918-e9a5-4966-ad38-7af3ae4a2bec/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0bb13fe7-8a37-41d7-90aa-fed271998088/volumes/kubernetes.io~projected/kube-api-access-jn5xx DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/59d544f8-cac5-4693-b82e-13f2ca910bfe/volumes/kubernetes.io~projected/kube-api-access-sgtzl DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d5170510-91aa-451e-bc7d-e18c3aaca68b/volumes/kubernetes.io~projected/kube-api-access-rlprd DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1accce46-e302-4151-824a-5d9232a1be2b/volumes/kubernetes.io~projected/kube-api-access-jg72b DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/14ef22c78b45926184f1ff02ee96b2def3755aaea1a9e1114f1c7e284fa44d1a/userdata/shm DeviceMajor:0 DeviceMinor:69 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c8f3d51d1799a83759828681971124ce841d8a032e9f04d9497febb5277a8ff/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5b7cfbc5319e7f442eea768b44a14b523b9e5edfd84298635a40d58ca3b39671/userdata/shm DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e4e7ca17a2a73713d132705c2942e4e37f78afab4691539f23103ded1c329067/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/56a70dd10da311078ba4997ed73aeda78169deebedf9de4447bccda37f4d2d5a/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/168dd325ea4c992d8739825170c9cabf4d9d930c68c308b377d787bc1b159b57/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:145318213e86866 MacAddress:3e:54:93:24:1b:e2 Speed:10000 Mtu:8901} {Name:566c550e6b2d4a9 MacAddress:2a:b4:e6:0f:66:53 Speed:10000 Mtu:8901} {Name:9248a18ec011c2b MacAddress:b2:4d:ae:bc:f0:f7 Speed:10000 Mtu:8901} {Name:a949307fd59b4a3 MacAddress:96:a5:44:53:47:cd Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c6:37:e2:49 Speed:0 Mtu:9001} {Name:br-int MacAddress:6a:6c:88:e1:49:37 Speed:0 Mtu:8901} {Name:e3d18ddc87b1780 MacAddress:b2:5f:98:17:8a:47 Speed:10000 Mtu:8901} {Name:eccd84d9e3c2a2a MacAddress:c6:37:13:56:82:f4 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c6:37:e2:49 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:aa:00:3b:6c:eb:14 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:4e:75:1e:6f:e4:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966201 4524 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966241 4524 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966529 4524 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966550 4524 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-171.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966665 4524 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966673 4524 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966686 4524 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966698 4524 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966773 4524 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.966918 4524 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.967014 4524 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.967045 4524 kubelet.go:493] "Attempting to sync node with API server" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.967054 4524 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.967066 4524 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.967074 4524 kubelet.go:405] "Adding apiserver pod source" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.967082 4524 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.967662 4524 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.967671 4524 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 03:04:16.969133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.968088 4524 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 03:04:16.970170 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970008 4524 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 03:04:16.970170 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970017 4524 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 03:04:16.970170 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970106 4524 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 03:04:16.970329 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970314 4524 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 03:04:16.970391 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970344 4524 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 03:04:16.970436 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970432 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 03:04:16.970481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970443 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 03:04:16.970481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970450 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 03:04:16.970481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970456 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 03:04:16.970481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970461 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 03:04:16.970481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970467 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 03:04:16.970481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970472 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 03:04:16.970481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970478 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 03:04:16.970937 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970506 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 03:04:16.970937 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970527 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 03:04:16.970937 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970537 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 03:04:16.970937 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970547 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 03:04:16.970937 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970571 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 03:04:16.970937 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.970577 4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 03:04:16.973754 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.973741 4524 server.go:1263] "Started kubelet" Oct 30 03:04:16.973867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.973811 4524 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 03:04:16.973933 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.973853 4524 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 03:04:16.973984 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.973955 4524 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 03:04:16.974277 ip-10-0-132-171 systemd[1]: Started Kubernetes Kubelet. Oct 30 03:04:16.974545 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.974499 4524 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 03:04:16.975133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.975118 4524 server.go:317] "Adding debug handlers to kubelet server" Oct 30 03:04:16.978169 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.977025 4524 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 03:04:16.978349 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.978153 4524 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 03:04:16.978423 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.978242 4524 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 03:04:16.978423 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.978375 4524 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 03:04:16.978423 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:16.978383 4524 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-171.ec2.internal\" not found" Oct 30 03:04:16.978599 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.978540 4524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 02:58:38 +0000 UTC" deadline="2028-07-28 17:38:30.22606762 +0000 UTC" Oct 30 03:04:16.978599 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.978594 4524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15302h34m13.247478622s" Oct 30 03:04:16.978691 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.978255 4524 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 03:04:16.979327 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.979246 4524 factory.go:55] Registering systemd factory Oct 30 03:04:16.979327 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.979264 4524 factory.go:223] Registration of the systemd container factory successfully Oct 30 03:04:16.983956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.983826 4524 factory.go:153] Registering CRI-O factory Oct 30 03:04:16.983956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.983850 4524 factory.go:223] Registration of the crio container factory successfully Oct 30 03:04:16.983956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.983904 4524 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 03:04:16.983956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.983927 4524 factory.go:103] Registering Raw factory Oct 30 03:04:16.983956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.983941 4524 manager.go:1199] Started watching for new ooms in manager Oct 30 03:04:16.984279 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984264 4524 manager.go:319] Starting recovery of all containers Oct 30 03:04:16.984661 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984625 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdb6477c-e16b-451c-84bd-8e62d3efc79f" volumeName="kubernetes.io/secret/bdb6477c-e16b-451c-84bd-8e62d3efc79f-cert" seLinuxMountContext="" Oct 30 03:04:16.984736 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984666 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="da1b6928-2507-4293-8e3c-f548c5b8ffab" volumeName="kubernetes.io/configmap/da1b6928-2507-4293-8e3c-f548c5b8ffab-iptables-alerter-script" seLinuxMountContext="" Oct 30 03:04:16.984736 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984680 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" volumeName="kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path" seLinuxMountContext="" Oct 30 03:04:16.984736 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984693 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24" volumeName="kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-config" seLinuxMountContext="" Oct 30 03:04:16.984736 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984705 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b700fd45-b0e4-4cf2-a2d7-ed70c389bed8" volumeName="kubernetes.io/projected/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-kube-api-access-s42qh" seLinuxMountContext="" Oct 30 03:04:16.984736 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984718 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="be768a88-5dd3-44ff-800f-cd6603784ab4" volumeName="kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm" seLinuxMountContext="" Oct 30 03:04:16.984736 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984729 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5170510-91aa-451e-bc7d-e18c3aaca68b" volumeName="kubernetes.io/empty-dir/d5170510-91aa-451e-bc7d-e18c3aaca68b-tmp-dir" seLinuxMountContext="" Oct 30 03:04:16.984736 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984741 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1078aa9-08f9-482d-b68c-16163edfaad8" volumeName="kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-binary-copy" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984758 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bb13fe7-8a37-41d7-90aa-fed271998088" volumeName="kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-cni-binary-copy" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984771 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1accce46-e302-4151-824a-5d9232a1be2b" volumeName="kubernetes.io/projected/1accce46-e302-4151-824a-5d9232a1be2b-kube-api-access-jg72b" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984782 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31deab60-7ba7-431c-9fdd-f15388845315" volumeName="kubernetes.io/projected/31deab60-7ba7-431c-9fdd-f15388845315-kube-api-access-6nhkc" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984796 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" volumeName="kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984807 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4abe5e4c-87c7-48d4-844e-97bd0ec1c99f" volumeName="kubernetes.io/projected/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-api-access-dx9m2" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984819 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24" volumeName="kubernetes.io/secret/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984833 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4abe5e4c-87c7-48d4-844e-97bd0ec1c99f" volumeName="kubernetes.io/empty-dir/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-data-volume" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984844 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="59d544f8-cac5-4693-b82e-13f2ca910bfe" volumeName="kubernetes.io/projected/59d544f8-cac5-4693-b82e-13f2ca910bfe-kube-api-access-sgtzl" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984857 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24" volumeName="kubernetes.io/projected/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-kube-api-access-6g62l" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984870 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1078aa9-08f9-482d-b68c-16163edfaad8" volumeName="kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984882 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1078aa9-08f9-482d-b68c-16163edfaad8" volumeName="kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984894 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bb13fe7-8a37-41d7-90aa-fed271998088" volumeName="kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-daemon-config" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984949 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="983ff918-e9a5-4966-ad38-7af3ae4a2bec" volumeName="kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-tuned" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984964 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="983ff918-e9a5-4966-ad38-7af3ae4a2bec" volumeName="kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-tmp" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984977 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1078aa9-08f9-482d-b68c-16163edfaad8" volumeName="kubernetes.io/projected/f1078aa9-08f9-482d-b68c-16163edfaad8-kube-api-access-wlncr" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.984997 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bac3d0d-dded-4047-81d5-333e47612f93" volumeName="kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985009 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24" volumeName="kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-script-lib" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985020 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a093097-53a0-40a5-9152-0e2930424cec" volumeName="kubernetes.io/secret/9a093097-53a0-40a5-9152-0e2930424cec-agent-certs" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985046 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdb6477c-e16b-451c-84bd-8e62d3efc79f" volumeName="kubernetes.io/projected/bdb6477c-e16b-451c-84bd-8e62d3efc79f-kube-api-access-xlj67" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985067 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="da1b6928-2507-4293-8e3c-f548c5b8ffab" volumeName="kubernetes.io/projected/da1b6928-2507-4293-8e3c-f548c5b8ffab-kube-api-access-nh86z" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985083 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bb13fe7-8a37-41d7-90aa-fed271998088" volumeName="kubernetes.io/projected/0bb13fe7-8a37-41d7-90aa-fed271998088-kube-api-access-jn5xx" seLinuxMountContext="" Oct 30 03:04:16.985086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985096 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45" volumeName="kubernetes.io/configmap/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-serviceca" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985106 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" volumeName="kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985119 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24" volumeName="kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-env-overrides" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985130 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a093097-53a0-40a5-9152-0e2930424cec" volumeName="kubernetes.io/configmap/9a093097-53a0-40a5-9152-0e2930424cec-konnectivity-ca" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985138 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b700fd45-b0e4-4cf2-a2d7-ed70c389bed8" volumeName="kubernetes.io/configmap/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-cabundle" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985146 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31deab60-7ba7-431c-9fdd-f15388845315" volumeName="kubernetes.io/empty-dir/31deab60-7ba7-431c-9fdd-f15388845315-tmp-dir" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985154 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b700fd45-b0e4-4cf2-a2d7-ed70c389bed8" volumeName="kubernetes.io/secret/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-key" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985161 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5170510-91aa-451e-bc7d-e18c3aaca68b" volumeName="kubernetes.io/configmap/d5170510-91aa-451e-bc7d-e18c3aaca68b-config-volume" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985168 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5170510-91aa-451e-bc7d-e18c3aaca68b" volumeName="kubernetes.io/projected/d5170510-91aa-451e-bc7d-e18c3aaca68b-kube-api-access-rlprd" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985174 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45" volumeName="kubernetes.io/projected/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-kube-api-access-6hm8w" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985181 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" volumeName="kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985188 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="662545a9-fe29-4583-8fdd-8ce384ee6681" volumeName="kubernetes.io/projected/662545a9-fe29-4583-8fdd-8ce384ee6681-kube-api-access-2zrc6" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985195 4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="983ff918-e9a5-4966-ad38-7af3ae4a2bec" volumeName="kubernetes.io/projected/983ff918-e9a5-4966-ad38-7af3ae4a2bec-kube-api-access-wptw5" seLinuxMountContext="" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985202 4524 reconstruct.go:97] "Volume reconstruction finished" Oct 30 03:04:16.986007 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:16.985207 4524 reconciler.go:29] "Reconciler: start to sync state" Oct 30 03:04:17.012617 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.012573 4524 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 03:04:17.014177 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.014145 4524 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 03:04:17.014177 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.014173 4524 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 03:04:17.014346 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.014205 4524 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 03:04:17.014346 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:17.014244 4524 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 03:04:17.035656 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.035633 4524 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:04:17.035786 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.035705 4524 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 03:04:17.039107 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.039081 4524 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="1f22696ed015bf99bf1e9eb1d8af91d259dc3a0074638c6684f89e65a60104c3" exitCode=0 Oct 30 03:04:17.039107 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.039105 4524 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="8d4a1595f67e6bbdaeeb2d0e098c0f0d132aca11f400c3f99fc401cb99c76122" exitCode=0 Oct 30 03:04:17.039275 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.039113 4524 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="882f7bb8a0b3344702988bbc7b31b8440fc60bed93532ca48a9ef2f02d71f989" exitCode=0 Oct 30 03:04:17.039275 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.039120 4524 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="61488809824a71c8b859f9dc7b6ef7acdbff57888ba5298e6bbbcf770ff7afef" exitCode=0 Oct 30 03:04:17.039275 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.039125 4524 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="5e970f7ea1f2ab88b086aa640472f9da5f8a245629100b0fa65186dbf58b217b" exitCode=0 Oct 30 03:04:17.039275 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.039131 4524 generic.go:356] "Generic (PLEG): container finished" podID="f1078aa9-08f9-482d-b68c-16163edfaad8" containerID="1be8f45d0b2e5bd05ac7bd6b684f6f0d62ad4309d0d556d580991f1314f3b8e0" exitCode=0 Oct 30 03:04:17.040013 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.039994 4524 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:04:17.049339 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.049282 4524 generic.go:356] "Generic (PLEG): container finished" podID="9c72cf2f385a5a6399c0c2530d39d3b9" containerID="e158f69ccd1227e481c49b96e4423015e1f06845604968a0ba6955ae0710f9ef" exitCode=0 Oct 30 03:04:17.053362 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.053344 4524 manager.go:324] Recovery completed Oct 30 03:04:17.054126 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.054109 4524 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:04:17.054263 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.054246 4524 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 03:04:17.067469 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067453 4524 cpu_manager.go:226] "Starting" policy="none" Oct 30 03:04:17.067469 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067465 4524 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 03:04:17.067588 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067480 4524 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 03:04:17.067621 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067602 4524 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 03:04:17.067651 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067613 4524 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 03:04:17.067683 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067651 4524 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 03:04:17.067683 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067656 4524 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 03:04:17.067683 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067662 4524 policy_none.go:50] "Start" Oct 30 03:04:17.067683 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067670 4524 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 03:04:17.067683 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067679 4524 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 03:04:17.067814 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067792 4524 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 03:04:17.067814 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067798 4524 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 03:04:17.067814 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.067804 4524 policy_none.go:44] "Start" Oct 30 03:04:17.071915 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.071891 4524 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 03:04:17.074607 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.074589 4524 manager.go:341] "Starting Device Plugin manager" Oct 30 03:04:17.075253 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:17.075234 4524 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 03:04:17.075306 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.075256 4524 server.go:87] "Starting device plugin registration server" Oct 30 03:04:17.075578 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.075567 4524 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 03:04:17.075624 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.075581 4524 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 03:04:17.075720 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.075707 4524 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 03:04:17.075822 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.075792 4524 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 03:04:17.075905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.075829 4524 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 03:04:17.075905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.075836 4524 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 03:04:17.076047 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.075876 4524 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T03:04:17.075819351Z","UUID":"3aaa5f48-b32f-4884-98fe-6de83eeafd96","Handler":null,"Name":"","Endpoint":""} Oct 30 03:04:17.077585 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.077567 4524 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 03:04:17.077685 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.077595 4524 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 03:04:17.080160 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:17.080140 4524 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 03:04:17.115142 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.115108 4524 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal"] Oct 30 03:04:17.115649 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.115618 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" event={"ID":"0e5cbe062ddc538d97b4fdc348b9dc76","Type":"ContainerStarted","Data":"285df8f8d4e3e831df9470a15075191991b81948bbd017b973b7327b673e0afa"} Oct 30 03:04:17.115733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.115660 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" event={"ID":"0e5cbe062ddc538d97b4fdc348b9dc76","Type":"ContainerStarted","Data":"74617c2f962a27e1da516a2cc4d931a206d42110349bc68969a6e6402fcaaaaa"} Oct 30 03:04:17.115733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.115722 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" event={"ID":"9c72cf2f385a5a6399c0c2530d39d3b9","Type":"ContainerStarted","Data":"fd8848ad4841b5d861aa42812a14653e7a81579079340eee7820c3fbd55cb2b6"} Oct 30 03:04:17.115840 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.115743 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" event={"ID":"9c72cf2f385a5a6399c0c2530d39d3b9","Type":"ContainerDied","Data":"e158f69ccd1227e481c49b96e4423015e1f06845604968a0ba6955ae0710f9ef"} Oct 30 03:04:17.115840 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.115762 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" event={"ID":"9c72cf2f385a5a6399c0c2530d39d3b9","Type":"ContainerStarted","Data":"e694b13bb4c8ba60b7ab3c256513262d4a8cc5d4cc547880d9254dd9a9094d3d"} Oct 30 03:04:17.115840 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.115780 4524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.115840 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.115811 4524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.166220 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.166198 4524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 03:04:17.166326 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:17.166239 4524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-171.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.168040 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.168026 4524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 03:04:17.168098 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:17.168051 4524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.176229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.176211 4524 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 03:04:17.177434 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.177417 4524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientMemory" Oct 30 03:04:17.177499 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.177444 4524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 03:04:17.177499 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.177455 4524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-171.ec2.internal" event="NodeHasSufficientPID" Oct 30 03:04:17.177499 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.177486 4524 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.236424 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.236399 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e5cbe062ddc538d97b4fdc348b9dc76-config\") pod \"kube-apiserver-proxy-ip-10-0-132-171.ec2.internal\" (UID: \"0e5cbe062ddc538d97b4fdc348b9dc76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.236533 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.236425 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.236533 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.236443 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.246477 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.246456 4524 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.246559 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.246510 4524 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.336989 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.336967 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e5cbe062ddc538d97b4fdc348b9dc76-config\") pod \"kube-apiserver-proxy-ip-10-0-132-171.ec2.internal\" (UID: \"0e5cbe062ddc538d97b4fdc348b9dc76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.337086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.336926 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0e5cbe062ddc538d97b4fdc348b9dc76-config\") pod \"kube-apiserver-proxy-ip-10-0-132-171.ec2.internal\" (UID: \"0e5cbe062ddc538d97b4fdc348b9dc76\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.337086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.337036 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.337086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.337053 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.337183 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.337088 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.337183 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.337117 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9c72cf2f385a5a6399c0c2530d39d3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" (UID: \"9c72cf2f385a5a6399c0c2530d39d3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:17.968220 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:17.968186 4524 apiserver.go:52] "Watching apiserver" Oct 30 03:04:18.018813 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.018736 4524 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 03:04:18.020044 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.020003 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-klqwb","openshift-network-diagnostics/network-check-target-8cffw","kube-system/konnectivity-agent-plrrl","openshift-image-registry/node-ca-5pk5f","openshift-insights/insights-runtime-extractor-d6fwr","openshift-insights/periodic-gathering-wbbgs-f2qnw","openshift-ovn-kubernetes/ovnkube-node-lwxh9","openshift-ingress-canary/ingress-canary-b4nrx","openshift-multus/multus-additional-cni-plugins-wk25h","kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal","openshift-multus/multus-tbms7","openshift-service-ca/service-ca-5fffc97bdf-mg496","openshift-cluster-node-tuning-operator/tuned-jwjmm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq","openshift-dns/dns-default-nxn6c","openshift-dns/node-resolver-tslzq","kube-system/global-pull-secret-syncer-2p9zt","openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2","openshift-multus/network-metrics-daemon-ks76k"] Oct 30 03:04:18.023574 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.023552 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:18.026426 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.026410 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.028225 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.028210 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.042292 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.042261 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 03:04:18.042527 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.042501 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wmngf\"" Oct 30 03:04:18.042603 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.042593 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5z89n\"" Oct 30 03:04:18.042663 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.042599 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 03:04:18.042663 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.042609 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-zrtj5\"" Oct 30 03:04:18.043565 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.043551 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lplr6\"" Oct 30 03:04:18.043817 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.043802 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 03:04:18.043916 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.043902 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 03:04:18.043965 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.043901 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rrhnz\"" Oct 30 03:04:18.044010 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.043973 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qvz6h\"" Oct 30 03:04:18.044105 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044094 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 03:04:18.044133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044099 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.044133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044117 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 03:04:18.044189 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044131 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.044547 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044532 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.044602 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044555 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.044602 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044572 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.044602 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044595 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 03:04:18.044698 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.044610 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.055234 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.055212 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.055336 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.055258 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 03:04:18.055336 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.055288 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 03:04:18.055712 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.055691 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.055806 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.055741 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 03:04:18.056460 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.056445 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mhf7m\"" Oct 30 03:04:18.056943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.056927 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h8g5n\"" Oct 30 03:04:18.057042 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.056963 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-5jrfh\"" Oct 30 03:04:18.057042 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.057023 4524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:04:18.057140 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.057101 4524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:18.057318 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.057241 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 03:04:18.057473 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.057459 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-mq4br\"" Oct 30 03:04:18.057593 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.057568 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zjwd2\"" Oct 30 03:04:18.058054 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058037 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 03:04:18.058112 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058069 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 03:04:18.058158 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058106 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 03:04:18.058158 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058076 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-676kj\"" Oct 30 03:04:18.058256 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058237 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 03:04:18.058310 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058238 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.058310 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058240 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.058310 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058293 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 03:04:18.058454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058315 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.058454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058335 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 03:04:18.058454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058448 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 03:04:18.058733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058447 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 03:04:18.058733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058478 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.058733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058502 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 03:04:18.058733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058478 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 03:04:18.058733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058583 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-x2bfb\"" Oct 30 03:04:18.058733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058575 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nxm67\"" Oct 30 03:04:18.058733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058652 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.058733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058686 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.058984 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058681 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-g6lgw\"" Oct 30 03:04:18.058984 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058844 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.058984 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058763 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.058984 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058762 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.058984 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.058928 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 03:04:18.059171 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059039 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jsmnt\"" Oct 30 03:04:18.059171 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059045 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2x78k\"" Oct 30 03:04:18.059171 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059058 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 03:04:18.059171 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059061 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.059171 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059095 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rwqxl\"" Oct 30 03:04:18.059324 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059097 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.059591 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059570 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.059824 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059805 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.059902 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.059887 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.060077 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.060059 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 03:04:18.060238 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.060225 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 03:04:18.071980 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.071962 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 03:04:18.079857 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.079842 4524 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 03:04:18.083624 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.083610 4524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 03:04:18.083689 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:18.083646 4524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-171.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-171.ec2.internal" Oct 30 03:04:18.085006 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.084985 4524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 03:04:18.085070 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:18.085031 4524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-171.ec2.internal" Oct 30 03:04:18.141705 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141670 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-etc-selinux\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.141705 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141709 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5170510-91aa-451e-bc7d-e18c3aaca68b-tmp-dir\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.141903 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141726 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.141903 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141845 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.141903 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141877 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-cabundle\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:18.141903 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141888 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5170510-91aa-451e-bc7d-e18c3aaca68b-tmp-dir\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.141903 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141896 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-tmp\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.142061 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141915 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.142061 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141939 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-systemd-units\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.142061 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141977 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-tmp\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.142061 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.141994 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.142061 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142022 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a093097-53a0-40a5-9152-0e2930424cec-konnectivity-ca\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:04:18.142061 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142044 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh86z\" (UniqueName: \"kubernetes.io/projected/da1b6928-2507-4293-8e3c-f548c5b8ffab-kube-api-access-nh86z\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:04:18.142291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142077 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-run\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.142291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142104 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nhkc\" (UniqueName: \"kubernetes.io/projected/31deab60-7ba7-431c-9fdd-f15388845315-kube-api-access-6nhkc\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:04:18.142291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142108 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.142291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142113 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-cabundle\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:18.142291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142129 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-host\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:04:18.142291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142152 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-conf\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.142291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142177 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-multus\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.142291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142199 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-hostroot\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142304 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142350 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-etc-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142368 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142382 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a093097-53a0-40a5-9152-0e2930424cec-konnectivity-ca\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142392 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142423 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-bin\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142438 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysconfig\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142453 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-cnibin\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142499 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31deab60-7ba7-431c-9fdd-f15388845315-hosts-file\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142535 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-etc-kubernetes\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142558 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-registration-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142603 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-device-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142659 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142691 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g62l\" (UniqueName: \"kubernetes.io/projected/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-kube-api-access-6g62l\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142710 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg72b\" (UniqueName: \"kubernetes.io/projected/1accce46-e302-4151-824a-5d9232a1be2b-kube-api-access-jg72b\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142829 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142858 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-k8s-cni-cncf-io\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.142876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142881 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-multus-certs\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142905 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-env-overrides\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.142931 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-serviceca\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143008 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da1b6928-2507-4293-8e3c-f548c5b8ffab-iptables-alerter-script\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143036 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s42qh\" (UniqueName: \"kubernetes.io/projected/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-kube-api-access-s42qh\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143063 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-modprobe-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143082 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-system-cni-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143087 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-env-overrides\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143061 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143104 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31deab60-7ba7-431c-9fdd-f15388845315-tmp-dir\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143175 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31deab60-7ba7-431c-9fdd-f15388845315-tmp-dir\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143175 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-socket-dir-parent\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143197 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-serviceca\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143219 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-kubelet\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143235 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da1b6928-2507-4293-8e3c-f548c5b8ffab-iptables-alerter-script\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143261 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-systemd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143338 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-ovn\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143379 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143407 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlprd\" (UniqueName: \"kubernetes.io/projected/d5170510-91aa-451e-bc7d-e18c3aaca68b-kube-api-access-rlprd\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143422 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-kubelet\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143454 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143554 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-log-socket\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143591 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-key\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143609 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlncr\" (UniqueName: \"kubernetes.io/projected/f1078aa9-08f9-482d-b68c-16163edfaad8-kube-api-access-wlncr\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143627 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-cnibin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.143635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143644 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-crio-socket\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143671 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143676 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-var-lib-kubelet\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143784 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-signing-key\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143793 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wptw5\" (UniqueName: \"kubernetes.io/projected/983ff918-e9a5-4966-ad38-7af3ae4a2bec-kube-api-access-wptw5\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143918 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-kubelet-config\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143958 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-dbus\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.143994 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144026 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144118 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144135 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-system-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144172 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-node-log\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144198 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1bac3d0d-dded-4047-81d5-333e47612f93-original-pull-secret\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144262 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a093097-53a0-40a5-9152-0e2930424cec-agent-certs\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144273 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1078aa9-08f9-482d-b68c-16163edfaad8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144287 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-socket-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144314 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-sys\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144330 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-netns\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144346 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xntc\" (UniqueName: \"kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144361 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-data-volume\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144375 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-netns\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144426 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-data-volume\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144421 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-var-lib-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144468 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144496 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovn-node-metrics-cert\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144531 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a093097-53a0-40a5-9152-0e2930424cec-agent-certs\") pod \"konnectivity-agent-plrrl\" (UID: \"9a093097-53a0-40a5-9152-0e2930424cec\") " pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144536 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1b6928-2507-4293-8e3c-f548c5b8ffab-host-slash\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144609 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-kubernetes\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.144614 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144630 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdb6477c-e16b-451c-84bd-8e62d3efc79f-cert\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144653 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-cni-binary-copy\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144662 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovn-node-metrics-cert\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144677 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-config\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144695 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hm8w\" (UniqueName: \"kubernetes.io/projected/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-kube-api-access-6hm8w\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144713 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-sys-fs\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144761 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-systemd\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144865 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdb6477c-e16b-451c-84bd-8e62d3efc79f-cert\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144862 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-tuned\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144914 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144941 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-tuned\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144957 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-config\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144942 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-conf-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144964 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-cni-binary-copy\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.144987 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145011 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-netd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145027 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5170510-91aa-451e-bc7d-e18c3aaca68b-config-volume\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145042 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-daemon-config\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145057 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn5xx\" (UniqueName: \"kubernetes.io/projected/0bb13fe7-8a37-41d7-90aa-fed271998088-kube-api-access-jn5xx\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145090 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zrc6\" (UniqueName: \"kubernetes.io/projected/662545a9-fe29-4583-8fdd-8ce384ee6681-kube-api-access-2zrc6\") pod \"migrator-66df5d496b-pdqh2\" (UID: \"662545a9-fe29-4583-8fdd-8ce384ee6681\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145168 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145202 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145228 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-script-lib\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145261 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-lib-modules\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145288 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-host\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145310 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-daemon-config\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145304 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d5170510-91aa-451e-bc7d-e18c3aaca68b-config-volume\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145347 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlj67\" (UniqueName: \"kubernetes.io/projected/bdb6477c-e16b-451c-84bd-8e62d3efc79f-kube-api-access-xlj67\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145386 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-bin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145483 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-ovnkube-script-lib\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145481 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145550 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgtzl\" (UniqueName: \"kubernetes.io/projected/59d544f8-cac5-4693-b82e-13f2ca910bfe-kube-api-access-sgtzl\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145569 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx9m2\" (UniqueName: \"kubernetes.io/projected/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-api-access-dx9m2\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.145625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145585 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-slash\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.146574 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.145780 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:18.175229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175053 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx9m2\" (UniqueName: \"kubernetes.io/projected/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-api-access-dx9m2\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.175229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175167 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nhkc\" (UniqueName: \"kubernetes.io/projected/31deab60-7ba7-431c-9fdd-f15388845315-kube-api-access-6nhkc\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:04:18.175229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175204 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn5xx\" (UniqueName: \"kubernetes.io/projected/0bb13fe7-8a37-41d7-90aa-fed271998088-kube-api-access-jn5xx\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.175543 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175204 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g62l\" (UniqueName: \"kubernetes.io/projected/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-kube-api-access-6g62l\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.175543 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175204 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wptw5\" (UniqueName: \"kubernetes.io/projected/983ff918-e9a5-4966-ad38-7af3ae4a2bec-kube-api-access-wptw5\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.175543 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175320 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlprd\" (UniqueName: \"kubernetes.io/projected/d5170510-91aa-451e-bc7d-e18c3aaca68b-kube-api-access-rlprd\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.175543 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175413 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s42qh\" (UniqueName: \"kubernetes.io/projected/b700fd45-b0e4-4cf2-a2d7-ed70c389bed8-kube-api-access-s42qh\") pod \"service-ca-5fffc97bdf-mg496\" (UID: \"b700fd45-b0e4-4cf2-a2d7-ed70c389bed8\") " pod="openshift-service-ca/service-ca-5fffc97bdf-mg496" Oct 30 03:04:18.175543 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175457 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh86z\" (UniqueName: \"kubernetes.io/projected/da1b6928-2507-4293-8e3c-f548c5b8ffab-kube-api-access-nh86z\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:04:18.175782 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.175672 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlncr\" (UniqueName: \"kubernetes.io/projected/f1078aa9-08f9-482d-b68c-16163edfaad8-kube-api-access-wlncr\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.176987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.176967 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xntc\" (UniqueName: \"kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc\") pod \"periodic-gathering-wbbgs-f2qnw\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:04:18.179987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.179965 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg72b\" (UniqueName: \"kubernetes.io/projected/1accce46-e302-4151-824a-5d9232a1be2b-kube-api-access-jg72b\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.180288 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.180272 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlj67\" (UniqueName: \"kubernetes.io/projected/bdb6477c-e16b-451c-84bd-8e62d3efc79f-kube-api-access-xlj67\") pod \"ingress-canary-b4nrx\" (UID: \"bdb6477c-e16b-451c-84bd-8e62d3efc79f\") " pod="openshift-ingress-canary/ingress-canary-b4nrx" Oct 30 03:04:18.180288 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.180280 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgtzl\" (UniqueName: \"kubernetes.io/projected/59d544f8-cac5-4693-b82e-13f2ca910bfe-kube-api-access-sgtzl\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:18.180387 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.180311 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bcnm\" (UniqueName: \"kubernetes.io/projected/be768a88-5dd3-44ff-800f-cd6603784ab4-kube-api-access-6bcnm\") pod \"network-check-target-8cffw\" (UID: \"be768a88-5dd3-44ff-800f-cd6603784ab4\") " pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:18.180387 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.180370 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zrc6\" (UniqueName: \"kubernetes.io/projected/662545a9-fe29-4583-8fdd-8ce384ee6681-kube-api-access-2zrc6\") pod \"migrator-66df5d496b-pdqh2\" (UID: \"662545a9-fe29-4583-8fdd-8ce384ee6681\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pdqh2" Oct 30 03:04:18.180616 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.180597 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hm8w\" (UniqueName: \"kubernetes.io/projected/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-kube-api-access-6hm8w\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:04:18.246331 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246295 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.246331 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246333 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246350 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-systemd-units\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246389 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246413 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-run\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246418 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246435 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-host\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246466 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246482 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-run\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:18.246475 4524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246458 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-conf\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246508 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45-host\") pod \"node-ca-5pk5f\" (UID: \"206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45\") " pod="openshift-image-registry/node-ca-5pk5f" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246432 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-systemd-units\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246563 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246551 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-multus\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246589 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-multus\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246591 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-hostroot\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:18.246633 4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:18.746609377 +0000 UTC m=+1.868175942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : secret "insights-runtime-extractor-tls" not found Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246634 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-hostroot\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246633 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-conf\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246686 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-etc-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246703 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246726 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-bin\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246736 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-etc-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246750 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysconfig\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246745 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246778 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-bin\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246783 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-cnibin\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246812 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysconfig\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246822 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31deab60-7ba7-431c-9fdd-f15388845315-hosts-file\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246845 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-etc-kubernetes\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246861 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-registration-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246875 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-cnibin\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246881 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-device-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246897 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-etc-kubernetes\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246897 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31deab60-7ba7-431c-9fdd-f15388845315-hosts-file\") pod \"node-resolver-tslzq\" (UID: \"31deab60-7ba7-431c-9fdd-f15388845315\") " pod="openshift-dns/node-resolver-tslzq" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246920 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-registration-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246925 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246923 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-device-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.246990 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.246990 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-k8s-cni-cncf-io\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247013 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-multus-certs\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247035 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-modprobe-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:18.246998 4524 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247054 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-system-cni-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247069 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-socket-dir-parent\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247084 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-kubelet\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:18.247107 4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls podName:d5170510-91aa-451e-bc7d-e18c3aaca68b nodeName:}" failed. No retries permitted until 2026-10-30 03:04:18.747091441 +0000 UTC m=+1.868657988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls") pod "dns-default-nxn6c" (UID: "d5170510-91aa-451e-bc7d-e18c3aaca68b") : secret "dns-default-metrics-tls" not found Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247098 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-k8s-cni-cncf-io\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247119 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-kubelet\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247129 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-multus-certs\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247137 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-system-cni-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247138 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-systemd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247179 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-systemd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247186 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-modprobe-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247187 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-socket-dir-parent\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247215 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-ovn\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247234 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-run-ovn\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247249 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247278 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-kubelet\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247300 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-log-socket\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247324 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-cnibin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247330 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-kubelet\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247348 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-crio-socket\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247358 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-sysctl-d\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247370 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-log-socket\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247371 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-var-lib-kubelet\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247401 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-crio-socket\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247410 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-kubelet-config\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247420 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-var-lib-kubelet\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247469 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-cnibin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247466 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-dbus\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247480 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-kubelet-config\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247502 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247535 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1bac3d0d-dded-4047-81d5-333e47612f93-dbus\") pod \"global-pull-secret-syncer-2p9zt\" (UID: \"1bac3d0d-dded-4047-81d5-333e47612f93\") " pod="kube-system/global-pull-secret-syncer-2p9zt" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247545 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-system-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247556 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1078aa9-08f9-482d-b68c-16163edfaad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wk25h\" (UID: \"f1078aa9-08f9-482d-b68c-16163edfaad8\") " pod="openshift-multus/multus-additional-cni-plugins-wk25h" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247569 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-node-log\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247593 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-socket-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247615 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-sys\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247637 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-netns\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247641 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-node-log\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247657 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-socket-dir\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247637 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-system-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247675 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-sys\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247678 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-run-netns\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247668 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-netns\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247701 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-run-netns\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247714 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-var-lib-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247730 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247750 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1b6928-2507-4293-8e3c-f548c5b8ffab-host-slash\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247774 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-kubernetes\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247776 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247778 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-var-lib-openvswitch\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247801 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-sys-fs\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247808 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da1b6928-2507-4293-8e3c-f548c5b8ffab-host-slash\") pod \"iptables-alerter-klqwb\" (UID: \"da1b6928-2507-4293-8e3c-f548c5b8ffab\") " pod="openshift-network-operator/iptables-alerter-klqwb" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247826 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-systemd\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247832 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-kubernetes\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247853 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-sys-fs\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247858 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247871 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-etc-systemd\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247880 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-conf-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247904 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247926 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-netd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247951 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-conf-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.247921 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247953 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247989 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-lib-modules\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248010 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-host\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248031 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-bin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.247990 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-cni-netd\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248055 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-slash\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248067 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-lib-modules\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248078 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-etc-selinux\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248084 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-multus-cni-dir\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248105 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bb13fe7-8a37-41d7-90aa-fed271998088-host-var-lib-cni-bin\") pod \"multus-tbms7\" (UID: \"0bb13fe7-8a37-41d7-90aa-fed271998088\") " pod="openshift-multus/multus-tbms7" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248105 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/983ff918-e9a5-4966-ad38-7af3ae4a2bec-host\") pod \"tuned-jwjmm\" (UID: \"983ff918-e9a5-4966-ad38-7af3ae4a2bec\") " pod="openshift-cluster-node-tuning-operator/tuned-jwjmm" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248138 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24-host-slash\") pod \"ovnkube-node-lwxh9\" (UID: \"9057bfb2-faf8-4a2b-bca6-3f1d5ab89f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248138 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1accce46-e302-4151-824a-5d9232a1be2b-etc-selinux\") pod \"aws-ebs-csi-driver-node-249bq\" (UID: \"1accce46-e302-4151-824a-5d9232a1be2b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-249bq" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248286 4524 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 03:04:18.249907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.248403 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.250428 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.250409 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d544f8-cac5-4693-b82e-13f2ca910bfe-metrics-certs\") pod \"network-metrics-daemon-ks76k\" (UID: \"59d544f8-cac5-4693-b82e-13f2ca910bfe\") " pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:18.324916 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.324882 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ks76k" Oct 30 03:04:18.635951 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.635920 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ks76k"] Oct 30 03:04:18.636820 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:18.636782 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d544f8_cac5_4693_b82e_13f2ca910bfe.slice/crio-e17daa4ebdc9a1fb3c4ff12164e1bcea901886a826e4f45442f70f978a4a8648 WatchSource:0}: Error finding container e17daa4ebdc9a1fb3c4ff12164e1bcea901886a826e4f45442f70f978a4a8648: Status 404 returned error can't find the container with id e17daa4ebdc9a1fb3c4ff12164e1bcea901886a826e4f45442f70f978a4a8648 Oct 30 03:04:18.638958 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.638938 4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 03:04:18.752315 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.752284 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:18.752452 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.752325 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.752452 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:18.752426 4524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 03:04:18.752556 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:18.752488 4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls podName:4abe5e4c-87c7-48d4-844e-97bd0ec1c99f nodeName:}" failed. No retries permitted until 2026-10-30 03:04:19.752474117 +0000 UTC m=+2.874040662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d6fwr" (UID: "4abe5e4c-87c7-48d4-844e-97bd0ec1c99f") : secret "insights-runtime-extractor-tls" not found Oct 30 03:04:18.754592 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.754566 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5170510-91aa-451e-bc7d-e18c3aaca68b-metrics-tls\") pod \"dns-default-nxn6c\" (UID: \"d5170510-91aa-451e-bc7d-e18c3aaca68b\") " pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:18.926785 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:18.926715 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:19.061187 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:19.061145 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ks76k" event={"ID":"59d544f8-cac5-4693-b82e-13f2ca910bfe","Type":"ContainerStarted","Data":"e17daa4ebdc9a1fb3c4ff12164e1bcea901886a826e4f45442f70f978a4a8648"} Oct 30 03:04:19.266381 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:19.266355 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nxn6c"] Oct 30 03:04:19.267200 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:19.267172 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5170510_91aa_451e_bc7d_e18c3aaca68b.slice/crio-66e658b9094b9977417ae22ba88bf4ff957cb54a7f360853065a78109c1a2171 WatchSource:0}: Error finding container 66e658b9094b9977417ae22ba88bf4ff957cb54a7f360853065a78109c1a2171: Status 404 returned error can't find the container with id 66e658b9094b9977417ae22ba88bf4ff957cb54a7f360853065a78109c1a2171 Oct 30 03:04:19.759093 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:19.759066 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:19.761451 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:19.761425 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4abe5e4c-87c7-48d4-844e-97bd0ec1c99f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d6fwr\" (UID: \"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f\") " pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:19.828734 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:19.828700 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d6fwr" Oct 30 03:04:20.066758 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:20.066718 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ks76k" event={"ID":"59d544f8-cac5-4693-b82e-13f2ca910bfe","Type":"ContainerStarted","Data":"31e5ab51817013d4ffdc98a8b550a8e153aee953ab8a6366d612fdf9ab84cbda"} Oct 30 03:04:20.067196 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:20.066780 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ks76k" event={"ID":"59d544f8-cac5-4693-b82e-13f2ca910bfe","Type":"ContainerStarted","Data":"7abda8405fe9fda25aa1a5138fb563b7ca4ea83848fd1f9ff3285b03dcfed93e"} Oct 30 03:04:20.067944 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:20.067923 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nxn6c" event={"ID":"d5170510-91aa-451e-bc7d-e18c3aaca68b","Type":"ContainerStarted","Data":"66e658b9094b9977417ae22ba88bf4ff957cb54a7f360853065a78109c1a2171"} Oct 30 03:04:20.146671 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:20.146642 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d6fwr"] Oct 30 03:04:20.148294 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:20.148269 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4abe5e4c_87c7_48d4_844e_97bd0ec1c99f.slice/crio-eaa57af3ab65bc89176e79b7dcf9392e26ea5777f155660eb5a3fedeb63794d3 WatchSource:0}: Error finding container eaa57af3ab65bc89176e79b7dcf9392e26ea5777f155660eb5a3fedeb63794d3: Status 404 returned error can't find the container with id eaa57af3ab65bc89176e79b7dcf9392e26ea5777f155660eb5a3fedeb63794d3 Oct 30 03:04:20.856549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:20.856506 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tslzq_31deab60-7ba7-431c-9fdd-f15388845315/dns-node-resolver/0.log" Oct 30 03:04:21.072594 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:21.072563 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d6fwr" event={"ID":"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f","Type":"ContainerStarted","Data":"769c9230668e2190263ea96b4d3e4c8f1ed07638a70bf5f12f6edaaafda3db91"} Oct 30 03:04:21.072950 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:21.072629 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d6fwr" event={"ID":"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f","Type":"ContainerStarted","Data":"75969db01ac890b1b4f36d5fc7fee97b6ce867b3864a2f4e0ee5e94e5bccafd0"} Oct 30 03:04:21.072950 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:21.072656 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d6fwr" event={"ID":"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f","Type":"ContainerStarted","Data":"eaa57af3ab65bc89176e79b7dcf9392e26ea5777f155660eb5a3fedeb63794d3"} Oct 30 03:04:21.074041 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:21.074006 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nxn6c" event={"ID":"d5170510-91aa-451e-bc7d-e18c3aaca68b","Type":"ContainerStarted","Data":"298c6346217d3a35005311af4ab8f57d8fffee4dd913ff286626ee9406d74022"} Oct 30 03:04:21.458694 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:21.458666 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5pk5f_206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45/node-ca/0.log" Oct 30 03:04:22.078265 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:22.078235 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nxn6c" event={"ID":"d5170510-91aa-451e-bc7d-e18c3aaca68b","Type":"ContainerStarted","Data":"88baf72da5f944063052a249b077f9f01025976481068a74ca05508c32788d99"} Oct 30 03:04:22.078752 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:22.078361 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:22.458670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:22.458594 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b4nrx_bdb6477c-e16b-451c-84bd-8e62d3efc79f/serve-healthcheck-canary/0.log" Oct 30 03:04:22.679968 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:22.679927 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:22.857159 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:22.857130 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-pdqh2_662545a9-fe29-4583-8fdd-8ce384ee6681/migrator/0.log" Oct 30 03:04:23.083561 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:23.083526 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d6fwr" event={"ID":"4abe5e4c-87c7-48d4-844e-97bd0ec1c99f","Type":"ContainerStarted","Data":"f183162b3292b6b52e290d70b193b5094ef4434e863aed8e7c9deb5f5c872e93"} Oct 30 03:04:26.419670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:26.419645 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:27.379550 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:27.379506 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:27.379728 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:27.379716 4524 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 03:04:31.682098 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:31.682069 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-plrrl" Oct 30 03:04:32.085788 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:32.085760 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nxn6c" Oct 30 03:04:33.022149 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.022113 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc"] Oct 30 03:04:33.025031 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.025008 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" Oct 30 03:04:33.036248 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.036224 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-ctqv4"] Oct 30 03:04:33.038963 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.038947 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:33.047275 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.047248 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc"] Oct 30 03:04:33.060099 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.060057 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lgm25\"" Oct 30 03:04:33.060304 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.060289 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Oct 30 03:04:33.067537 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.067501 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 03:04:33.067644 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.067568 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 03:04:33.068105 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.068086 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-ctqv4"] Oct 30 03:04:33.116780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.116760 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dd58cfc4-bczz9"] Oct 30 03:04:33.119700 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.119685 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.139061 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.139035 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq8dh\" (UniqueName: \"kubernetes.io/projected/44f95e92-057f-473e-beeb-641cabd274ff-kube-api-access-lq8dh\") pod \"downloads-6dd789c8b7-ctqv4\" (UID: \"44f95e92-057f-473e-beeb-641cabd274ff\") " pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:33.139170 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.139073 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed6376ac-50c3-4d9a-9fcc-707d164d6d75-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-jx2rc\" (UID: \"ed6376ac-50c3-4d9a-9fcc-707d164d6d75\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" Oct 30 03:04:33.139170 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.139115 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44f95e92-057f-473e-beeb-641cabd274ff-tmp\") pod \"downloads-6dd789c8b7-ctqv4\" (UID: \"44f95e92-057f-473e-beeb-641cabd274ff\") " pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:33.155570 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.155551 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 03:04:33.155816 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.155798 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 03:04:33.155943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.155802 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 03:04:33.155943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.155829 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 03:04:33.155943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.155870 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j969n\"" Oct 30 03:04:33.155943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.155943 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-s6vs2\"" Oct 30 03:04:33.156316 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.156299 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd58cfc4-bczz9"] Oct 30 03:04:33.240141 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240110 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-bound-sa-token\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.240141 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240143 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq8dh\" (UniqueName: \"kubernetes.io/projected/44f95e92-057f-473e-beeb-641cabd274ff-kube-api-access-lq8dh\") pod \"downloads-6dd789c8b7-ctqv4\" (UID: \"44f95e92-057f-473e-beeb-641cabd274ff\") " pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:33.240370 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240161 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44f95e92-057f-473e-beeb-641cabd274ff-tmp\") pod \"downloads-6dd789c8b7-ctqv4\" (UID: \"44f95e92-057f-473e-beeb-641cabd274ff\") " pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:33.240370 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240182 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1bd1743-b556-480a-a06d-9ee41a24c6d4-image-registry-private-configuration\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.240370 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240292 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1bd1743-b556-480a-a06d-9ee41a24c6d4-trusted-ca\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.240370 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240335 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1bd1743-b556-480a-a06d-9ee41a24c6d4-ca-trust-extracted\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.240370 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240355 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h2wk\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-kube-api-access-6h2wk\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.240621 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240374 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed6376ac-50c3-4d9a-9fcc-707d164d6d75-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-jx2rc\" (UID: \"ed6376ac-50c3-4d9a-9fcc-707d164d6d75\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" Oct 30 03:04:33.240621 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240394 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1bd1743-b556-480a-a06d-9ee41a24c6d4-registry-certificates\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.240621 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240477 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1bd1743-b556-480a-a06d-9ee41a24c6d4-installation-pull-secrets\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.240621 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240501 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-registry-tls\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.240621 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.240555 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44f95e92-057f-473e-beeb-641cabd274ff-tmp\") pod \"downloads-6dd789c8b7-ctqv4\" (UID: \"44f95e92-057f-473e-beeb-641cabd274ff\") " pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:33.242712 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.242697 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed6376ac-50c3-4d9a-9fcc-707d164d6d75-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-jx2rc\" (UID: \"ed6376ac-50c3-4d9a-9fcc-707d164d6d75\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" Oct 30 03:04:33.268476 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.268446 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq8dh\" (UniqueName: \"kubernetes.io/projected/44f95e92-057f-473e-beeb-641cabd274ff-kube-api-access-lq8dh\") pod \"downloads-6dd789c8b7-ctqv4\" (UID: \"44f95e92-057f-473e-beeb-641cabd274ff\") " pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:33.336068 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.336036 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" Oct 30 03:04:33.341587 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.341566 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1bd1743-b556-480a-a06d-9ee41a24c6d4-installation-pull-secrets\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.341690 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.341595 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-registry-tls\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.341690 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.341621 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-bound-sa-token\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.341890 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.341867 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1bd1743-b556-480a-a06d-9ee41a24c6d4-image-registry-private-configuration\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.341964 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.341946 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1bd1743-b556-480a-a06d-9ee41a24c6d4-trusted-ca\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.342017 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.341978 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1bd1743-b556-480a-a06d-9ee41a24c6d4-ca-trust-extracted\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.342017 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.342001 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h2wk\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-kube-api-access-6h2wk\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.342105 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.342032 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1bd1743-b556-480a-a06d-9ee41a24c6d4-registry-certificates\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.342404 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.342352 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1bd1743-b556-480a-a06d-9ee41a24c6d4-ca-trust-extracted\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.342914 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.342880 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1bd1743-b556-480a-a06d-9ee41a24c6d4-registry-certificates\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.343150 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.343125 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1bd1743-b556-480a-a06d-9ee41a24c6d4-trusted-ca\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.344266 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.344240 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1bd1743-b556-480a-a06d-9ee41a24c6d4-installation-pull-secrets\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.344337 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.344295 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1bd1743-b556-480a-a06d-9ee41a24c6d4-image-registry-private-configuration\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.344477 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.344458 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-registry-tls\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.355388 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.355361 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:33.369018 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.368997 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h2wk\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-kube-api-access-6h2wk\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.369168 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.369152 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1bd1743-b556-480a-a06d-9ee41a24c6d4-bound-sa-token\") pod \"image-registry-6dd58cfc4-bczz9\" (UID: \"c1bd1743-b556-480a-a06d-9ee41a24c6d4\") " pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.430901 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.430867 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:33.633456 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.633427 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc"] Oct 30 03:04:33.634168 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:33.634145 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6376ac_50c3_4d9a_9fcc_707d164d6d75.slice/crio-2c24f49b3cdcdf8e6b6b7bb5aa00f45a869928add22ab782e4cd144dc2677a89 WatchSource:0}: Error finding container 2c24f49b3cdcdf8e6b6b7bb5aa00f45a869928add22ab782e4cd144dc2677a89: Status 404 returned error can't find the container with id 2c24f49b3cdcdf8e6b6b7bb5aa00f45a869928add22ab782e4cd144dc2677a89 Oct 30 03:04:33.667740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.667714 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-ctqv4"] Oct 30 03:04:33.668341 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:33.668313 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f95e92_057f_473e_beeb_641cabd274ff.slice/crio-d9f411efdc69a372f705a95f2277ee95ad43c78f67c0dad7eced9fc8d1c188f2 WatchSource:0}: Error finding container d9f411efdc69a372f705a95f2277ee95ad43c78f67c0dad7eced9fc8d1c188f2: Status 404 returned error can't find the container with id d9f411efdc69a372f705a95f2277ee95ad43c78f67c0dad7eced9fc8d1c188f2 Oct 30 03:04:33.729225 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:33.729204 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd58cfc4-bczz9"] Oct 30 03:04:33.729555 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:33.729528 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1bd1743_b556_480a_a06d_9ee41a24c6d4.slice/crio-38de9deb3d97518b52148faacdc281946e6f75a40552f38f3dca33597140eb4c WatchSource:0}: Error finding container 38de9deb3d97518b52148faacdc281946e6f75a40552f38f3dca33597140eb4c: Status 404 returned error can't find the container with id 38de9deb3d97518b52148faacdc281946e6f75a40552f38f3dca33597140eb4c Oct 30 03:04:34.117793 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:34.117756 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" event={"ID":"c1bd1743-b556-480a-a06d-9ee41a24c6d4","Type":"ContainerStarted","Data":"ef2befd85dac9084ecb4b8d1a24dc55533af12fed7ce30005bf0590a2adc0f8f"} Oct 30 03:04:34.118220 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:34.117816 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" event={"ID":"c1bd1743-b556-480a-a06d-9ee41a24c6d4","Type":"ContainerStarted","Data":"38de9deb3d97518b52148faacdc281946e6f75a40552f38f3dca33597140eb4c"} Oct 30 03:04:34.118220 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:34.117844 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:04:34.119437 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:34.119386 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-ctqv4" event={"ID":"44f95e92-057f-473e-beeb-641cabd274ff","Type":"ContainerStarted","Data":"d9f411efdc69a372f705a95f2277ee95ad43c78f67c0dad7eced9fc8d1c188f2"} Oct 30 03:04:34.121073 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:34.121048 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" event={"ID":"ed6376ac-50c3-4d9a-9fcc-707d164d6d75","Type":"ContainerStarted","Data":"2c24f49b3cdcdf8e6b6b7bb5aa00f45a869928add22ab782e4cd144dc2677a89"} Oct 30 03:04:34.169533 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:34.169475 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" podStartSLOduration=1.16946441 podStartE2EDuration="1.16946441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:34.168994989 +0000 UTC m=+17.290561560" watchObservedRunningTime="2026-10-30 03:04:34.16946441 +0000 UTC m=+17.291030977" Oct 30 03:04:35.126919 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:35.126880 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" event={"ID":"ed6376ac-50c3-4d9a-9fcc-707d164d6d75","Type":"ContainerStarted","Data":"5efbfc0d5b1e5dc62d86c808a84fa7e52286af348cf084877cc78c010e47c759"} Oct 30 03:04:35.127385 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:35.127240 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" Oct 30 03:04:35.132304 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:35.132281 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" Oct 30 03:04:35.174167 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:35.174113 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-jx2rc" podStartSLOduration=2.143268801 podStartE2EDuration="3.174099397s" totalImagesPullingTime="1.030830596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:35.173665829 +0000 UTC m=+18.295232397" watchObservedRunningTime="2026-10-30 03:04:35.174099397 +0000 UTC m=+18.295665964" Oct 30 03:04:36.085000 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.084967 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk"] Oct 30 03:04:36.088128 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.088106 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.109401 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.109378 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk"] Oct 30 03:04:36.122806 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.122779 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 30 03:04:36.122954 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.122916 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 30 03:04:36.123273 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.123251 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rvr4b\"" Oct 30 03:04:36.123369 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.123331 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 03:04:36.123579 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.123448 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 03:04:36.123579 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.123548 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 03:04:36.265222 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.265186 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.265706 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.265237 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.265706 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.265337 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlsjd\" (UniqueName: \"kubernetes.io/projected/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-kube-api-access-zlsjd\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.265706 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.265369 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.366628 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.366557 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.366628 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.366611 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.366836 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.366643 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlsjd\" (UniqueName: \"kubernetes.io/projected/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-kube-api-access-zlsjd\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.366836 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.366662 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.367284 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.367261 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.369697 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.369679 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.369768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.369679 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.394686 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.394656 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlsjd\" (UniqueName: \"kubernetes.io/projected/efba1548-3e1c-403c-a9cf-d6f27b5b1a48-kube-api-access-zlsjd\") pod \"prometheus-operator-9cd89b74c-sr2lk\" (UID: \"efba1548-3e1c-403c-a9cf-d6f27b5b1a48\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.403314 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.403292 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" Oct 30 03:04:36.708264 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:36.708233 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk"] Oct 30 03:04:36.708855 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:36.708832 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefba1548_3e1c_403c_a9cf_d6f27b5b1a48.slice/crio-74887ffc39a031beea1b23fbdc1afe90d1918428246a3d8af3203b57c43f2907 WatchSource:0}: Error finding container 74887ffc39a031beea1b23fbdc1afe90d1918428246a3d8af3203b57c43f2907: Status 404 returned error can't find the container with id 74887ffc39a031beea1b23fbdc1afe90d1918428246a3d8af3203b57c43f2907 Oct 30 03:04:37.137406 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.137371 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" event={"ID":"efba1548-3e1c-403c-a9cf-d6f27b5b1a48","Type":"ContainerStarted","Data":"74887ffc39a031beea1b23fbdc1afe90d1918428246a3d8af3203b57c43f2907"} Oct 30 03:04:37.533232 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.533153 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-77cf5b8dc6-dkclh"] Oct 30 03:04:37.536353 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.536329 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.571182 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.571147 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 03:04:37.571311 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.571192 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 03:04:37.571383 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.571356 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 03:04:37.572409 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.572387 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77cf5b8dc6-dkclh"] Oct 30 03:04:37.574131 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.574086 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 03:04:37.574229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.574148 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 03:04:37.574229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.574159 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6wtc6\"" Oct 30 03:04:37.675596 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.675555 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-oauth-config\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.675596 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.675597 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-service-ca\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.675830 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.675714 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-oauth-serving-cert\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.675830 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.675751 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-serving-cert\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.675830 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.675790 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dftdx\" (UniqueName: \"kubernetes.io/projected/7317009d-02e3-4d2e-b8c8-13cf10b305e3-kube-api-access-dftdx\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.675830 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.675830 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7317009d-02e3-4d2e-b8c8-13cf10b305e3-tmp\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.676036 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.675878 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-config\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.776783 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.776743 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-serving-cert\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.776940 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.776794 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dftdx\" (UniqueName: \"kubernetes.io/projected/7317009d-02e3-4d2e-b8c8-13cf10b305e3-kube-api-access-dftdx\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.776940 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.776831 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7317009d-02e3-4d2e-b8c8-13cf10b305e3-tmp\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.776940 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.776867 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-config\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.776940 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.776910 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-oauth-config\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.776940 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.776935 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-service-ca\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.777207 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.776964 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-oauth-serving-cert\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.777363 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.777337 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7317009d-02e3-4d2e-b8c8-13cf10b305e3-tmp\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.777806 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.777783 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-oauth-serving-cert\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.777899 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.777806 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-service-ca\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.777899 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.777814 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-config\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.779772 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.779751 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-serving-cert\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.783165 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.783142 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-oauth-config\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.805121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.805100 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dftdx\" (UniqueName: \"kubernetes.io/projected/7317009d-02e3-4d2e-b8c8-13cf10b305e3-kube-api-access-dftdx\") pod \"console-77cf5b8dc6-dkclh\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:37.850792 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:37.850756 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:38.159607 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:38.159582 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77cf5b8dc6-dkclh"] Oct 30 03:04:38.168773 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:38.168614 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7317009d_02e3_4d2e_b8c8_13cf10b305e3.slice/crio-eb4c32bd7644f2d15634532077d4eb83f328fcdc36d258a4649cd7c4b72d4c12 WatchSource:0}: Error finding container eb4c32bd7644f2d15634532077d4eb83f328fcdc36d258a4649cd7c4b72d4c12: Status 404 returned error can't find the container with id eb4c32bd7644f2d15634532077d4eb83f328fcdc36d258a4649cd7c4b72d4c12 Oct 30 03:04:39.148331 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:39.147867 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" event={"ID":"efba1548-3e1c-403c-a9cf-d6f27b5b1a48","Type":"ContainerStarted","Data":"51b74e85dd1e9457289b5380c339cdf52df6a80c10273ba021dee29e692a1375"} Oct 30 03:04:39.148331 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:39.147928 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" event={"ID":"efba1548-3e1c-403c-a9cf-d6f27b5b1a48","Type":"ContainerStarted","Data":"b6aee2b2ead4b833e26fd22466563eae7327247351534a10448ed3ce4d0fbd89"} Oct 30 03:04:39.149161 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:39.149136 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77cf5b8dc6-dkclh" event={"ID":"7317009d-02e3-4d2e-b8c8-13cf10b305e3","Type":"ContainerStarted","Data":"eb4c32bd7644f2d15634532077d4eb83f328fcdc36d258a4649cd7c4b72d4c12"} Oct 30 03:04:39.197466 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:39.197409 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sr2lk" podStartSLOduration=1.80688832 podStartE2EDuration="3.197390054s" totalImagesPullingTime="1.390501734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:39.196301122 +0000 UTC m=+22.317867690" watchObservedRunningTime="2026-10-30 03:04:39.197390054 +0000 UTC m=+22.318956621" Oct 30 03:04:41.741975 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.741943 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jrzvm"] Oct 30 03:04:41.745810 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.745781 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.773419 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.773384 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 03:04:41.785965 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.785934 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 03:04:41.785965 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.785941 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-98cgx\"" Oct 30 03:04:41.815327 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.815305 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 03:04:41.906852 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.906822 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56b4051c-8b50-412e-bb8b-9a4ef725081e-metrics-client-ca\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.907034 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.906864 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.907034 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.906897 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-textfile\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.907152 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.907039 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-sys\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.907152 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.907070 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-root\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.907257 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.907169 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-wtmp\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.907257 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.907187 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxl47\" (UniqueName: \"kubernetes.io/projected/56b4051c-8b50-412e-bb8b-9a4ef725081e-kube-api-access-mxl47\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.907257 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.907206 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-tls\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:41.907257 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:41.907234 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007636 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56b4051c-8b50-412e-bb8b-9a4ef725081e-metrics-client-ca\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007694 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007729 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-textfile\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007760 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-sys\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007786 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-root\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007819 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-wtmp\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007847 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxl47\" (UniqueName: \"kubernetes.io/projected/56b4051c-8b50-412e-bb8b-9a4ef725081e-kube-api-access-mxl47\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007865 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-sys\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007879 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-tls\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.007937 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-root\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008636 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.008123 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-textfile\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008636 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.008172 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008636 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.008338 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008636 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.008346 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56b4051c-8b50-412e-bb8b-9a4ef725081e-metrics-client-ca\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.008636 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.008347 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-wtmp\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.010448 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.010428 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-tls\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.010821 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.010804 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56b4051c-8b50-412e-bb8b-9a4ef725081e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.035082 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.035055 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxl47\" (UniqueName: \"kubernetes.io/projected/56b4051c-8b50-412e-bb8b-9a4ef725081e-kube-api-access-mxl47\") pod \"node-exporter-jrzvm\" (UID: \"56b4051c-8b50-412e-bb8b-9a4ef725081e\") " pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.068530 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.068491 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jrzvm" Oct 30 03:04:42.081018 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:42.080991 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b4051c_8b50_412e_bb8b_9a4ef725081e.slice/crio-b4608aa6c2d605c1a80f8bd991855773857dcc5990e883bd60060afacb970b60 WatchSource:0}: Error finding container b4608aa6c2d605c1a80f8bd991855773857dcc5990e883bd60060afacb970b60: Status 404 returned error can't find the container with id b4608aa6c2d605c1a80f8bd991855773857dcc5990e883bd60060afacb970b60 Oct 30 03:04:42.161539 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.161488 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jrzvm" event={"ID":"56b4051c-8b50-412e-bb8b-9a4ef725081e","Type":"ContainerStarted","Data":"b4608aa6c2d605c1a80f8bd991855773857dcc5990e883bd60060afacb970b60"} Oct 30 03:04:42.163496 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.163468 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77cf5b8dc6-dkclh" event={"ID":"7317009d-02e3-4d2e-b8c8-13cf10b305e3","Type":"ContainerStarted","Data":"7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e"} Oct 30 03:04:42.213477 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.213433 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-77cf5b8dc6-dkclh" podStartSLOduration=2.228293127 podStartE2EDuration="5.213420701s" totalImagesPullingTime="2.985127574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:42.212494507 +0000 UTC m=+25.334061073" watchObservedRunningTime="2026-10-30 03:04:42.213420701 +0000 UTC m=+25.334987267" Oct 30 03:04:42.964324 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:42.963145 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lwxh9" Oct 30 03:04:43.170048 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:43.170013 4524 generic.go:356] "Generic (PLEG): container finished" podID="56b4051c-8b50-412e-bb8b-9a4ef725081e" containerID="f72050560c76cd60fc2e0dcba475e1cc87e6836b4d1f6e362167a93adafb3537" exitCode=0 Oct 30 03:04:43.170185 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:43.170116 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jrzvm" event={"ID":"56b4051c-8b50-412e-bb8b-9a4ef725081e","Type":"ContainerDied","Data":"f72050560c76cd60fc2e0dcba475e1cc87e6836b4d1f6e362167a93adafb3537"} Oct 30 03:04:44.176150 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:44.176115 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jrzvm" event={"ID":"56b4051c-8b50-412e-bb8b-9a4ef725081e","Type":"ContainerStarted","Data":"2c20a375bf5ea2b70591f6558f8141723c88476dc3c8609657697641fa352b09"} Oct 30 03:04:44.176615 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:44.176171 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jrzvm" event={"ID":"56b4051c-8b50-412e-bb8b-9a4ef725081e","Type":"ContainerStarted","Data":"ff7776c672e7636ed88288d49badbe91d809388d4000d131fb60f978ec771624"} Oct 30 03:04:44.255679 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:44.255632 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jrzvm" podStartSLOduration=2.272119426 podStartE2EDuration="3.255617712s" totalImagesPullingTime="904.427281ms" totalInitContainerRuntime="79.071005ms" isStatefulPod=true podCreationTimestamp="2026-10-30 03:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:44.240941385 +0000 UTC m=+27.362507954" watchObservedRunningTime="2026-10-30 03:04:44.255617712 +0000 UTC m=+27.377184279" Oct 30 03:04:47.851777 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:47.851740 4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:47.852252 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:47.851797 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:47.857276 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:47.857250 4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:48.194967 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:48.194883 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:04:49.341956 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:04:49.341921 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:04:51.204714 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.204677 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-ctqv4" event={"ID":"44f95e92-057f-473e-beeb-641cabd274ff","Type":"ContainerStarted","Data":"685e324f33596f0b08f9c9119b37c18778ca4de0e9c8aa8cd88db58828e05cb3"} Oct 30 03:04:51.205188 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.204736 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:51.221588 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.221565 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-ctqv4" Oct 30 03:04:51.283117 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.283061 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-ctqv4" podStartSLOduration=2.804499025 podStartE2EDuration="19.283046882s" totalImagesPullingTime="16.478547857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:51.2822304 +0000 UTC m=+34.403796969" watchObservedRunningTime="2026-10-30 03:04:51.283046882 +0000 UTC m=+34.404613448" Oct 30 03:04:51.586026 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.585995 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-767bf888d-lmtl2"] Oct 30 03:04:51.615381 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.615348 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767bf888d-lmtl2"] Oct 30 03:04:51.615582 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.615479 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.650162 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.650136 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 03:04:51.685589 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.685556 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-serving-cert\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.685757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.685604 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ce003de-7d0e-447f-adb3-a239839ecdc2-tmp\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.685757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.685632 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-oauth-config\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.685757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.685655 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-service-ca\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.685757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.685715 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-config\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.685757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.685739 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-oauth-serving-cert\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.686059 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.685764 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-trusted-ca-bundle\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.686059 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.685803 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r27cz\" (UniqueName: \"kubernetes.io/projected/6ce003de-7d0e-447f-adb3-a239839ecdc2-kube-api-access-r27cz\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.787042 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.786998 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-config\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.787239 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.787055 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-oauth-serving-cert\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.787239 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.787231 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-trusted-ca-bundle\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.787361 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.787277 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r27cz\" (UniqueName: \"kubernetes.io/projected/6ce003de-7d0e-447f-adb3-a239839ecdc2-kube-api-access-r27cz\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.787361 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.787312 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-serving-cert\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.787361 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.787346 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ce003de-7d0e-447f-adb3-a239839ecdc2-tmp\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.787504 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.787374 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-oauth-config\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.787504 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.787402 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-service-ca\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.788691 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.788667 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-oauth-serving-cert\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.788999 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.788953 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ce003de-7d0e-447f-adb3-a239839ecdc2-tmp\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.789107 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.789033 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-service-ca\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.789408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.789371 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-trusted-ca-bundle\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.789562 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.789538 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-config\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.791950 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.791926 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-serving-cert\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.794264 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.794223 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-oauth-config\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.816087 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.816065 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r27cz\" (UniqueName: \"kubernetes.io/projected/6ce003de-7d0e-447f-adb3-a239839ecdc2-kube-api-access-r27cz\") pod \"console-767bf888d-lmtl2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:51.952672 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:51.952592 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:04:52.294606 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:52.294471 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767bf888d-lmtl2"] Oct 30 03:04:52.296448 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:04:52.296418 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce003de_7d0e_447f_adb3_a239839ecdc2.slice/crio-edf47812b94545faad2c7231764fd078fb391943b04703f22c3c0c4ad47b8363 WatchSource:0}: Error finding container edf47812b94545faad2c7231764fd078fb391943b04703f22c3c0c4ad47b8363: Status 404 returned error can't find the container with id edf47812b94545faad2c7231764fd078fb391943b04703f22c3c0c4ad47b8363 Oct 30 03:04:52.682540 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:52.682491 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8cffw" Oct 30 03:04:53.214926 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:53.214885 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767bf888d-lmtl2" event={"ID":"6ce003de-7d0e-447f-adb3-a239839ecdc2","Type":"ContainerStarted","Data":"02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b"} Oct 30 03:04:53.215198 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:53.214949 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767bf888d-lmtl2" event={"ID":"6ce003de-7d0e-447f-adb3-a239839ecdc2","Type":"ContainerStarted","Data":"edf47812b94545faad2c7231764fd078fb391943b04703f22c3c0c4ad47b8363"} Oct 30 03:04:53.265505 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:53.265454 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-767bf888d-lmtl2" podStartSLOduration=2.265439318 podStartE2EDuration="2.265439318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:04:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:04:53.265029271 +0000 UTC m=+36.386595839" watchObservedRunningTime="2026-10-30 03:04:53.265439318 +0000 UTC m=+36.387005888" Oct 30 03:04:55.132301 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:04:55.132268 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dd58cfc4-bczz9" Oct 30 03:05:01.953635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:01.953548 4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:05:01.954217 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:01.953660 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:05:01.958424 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:01.958398 4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:05:02.249859 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:02.249791 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:05:02.334538 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:02.334500 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77cf5b8dc6-dkclh"] Oct 30 03:05:19.336566 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:05:19.336531 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:05:24.645631 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.645599 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-567b98f465-tjqxp"] Oct 30 03:05:24.705019 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.704995 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567b98f465-tjqxp"] Oct 30 03:05:24.705204 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.705089 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.816612 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.816577 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-serving-cert\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.816804 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.816622 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvnzh\" (UniqueName: \"kubernetes.io/projected/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-kube-api-access-kvnzh\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.816804 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.816698 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-oauth-serving-cert\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.816804 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.816735 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-tmp\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.816804 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.816791 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-service-ca\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.816943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.816850 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-oauth-config\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.816943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.816872 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-config\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.817006 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.816946 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-trusted-ca-bundle\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.917561 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917464 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-oauth-serving-cert\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.917561 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917500 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-tmp\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.917561 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917533 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-service-ca\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.917863 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917573 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-oauth-config\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.917863 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917590 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-config\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.917863 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917608 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-trusted-ca-bundle\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.917863 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917627 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-serving-cert\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.917863 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917665 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvnzh\" (UniqueName: \"kubernetes.io/projected/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-kube-api-access-kvnzh\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.918103 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.917927 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-tmp\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.918359 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.918336 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-config\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.918500 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.918460 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-service-ca\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.918653 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.918633 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-trusted-ca-bundle\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.918762 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.918743 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-oauth-serving-cert\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.920080 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.920054 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-oauth-config\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.920163 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.920090 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-console-serving-cert\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:24.944883 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:24.944862 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvnzh\" (UniqueName: \"kubernetes.io/projected/cf1e9f02-3ec0-42c3-939b-8e1017da6c47-kube-api-access-kvnzh\") pod \"console-567b98f465-tjqxp\" (UID: \"cf1e9f02-3ec0-42c3-939b-8e1017da6c47\") " pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:25.025757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:25.025735 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:25.368267 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:25.368240 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567b98f465-tjqxp"] Oct 30 03:05:25.368578 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:05:25.368553 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1e9f02_3ec0_42c3_939b_8e1017da6c47.slice/crio-183cfc561c8dea2c0af058de2272218455189deb726311ddb38dde156ecc7f65 WatchSource:0}: Error finding container 183cfc561c8dea2c0af058de2272218455189deb726311ddb38dde156ecc7f65: Status 404 returned error can't find the container with id 183cfc561c8dea2c0af058de2272218455189deb726311ddb38dde156ecc7f65 Oct 30 03:05:26.325079 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:26.325045 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567b98f465-tjqxp" event={"ID":"cf1e9f02-3ec0-42c3-939b-8e1017da6c47","Type":"ContainerStarted","Data":"0b9bd53d16a38734f068a1ae3148a9217480a9c2e71d5251cecccf899bb49e23"} Oct 30 03:05:26.325468 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:26.325099 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567b98f465-tjqxp" event={"ID":"cf1e9f02-3ec0-42c3-939b-8e1017da6c47","Type":"ContainerStarted","Data":"183cfc561c8dea2c0af058de2272218455189deb726311ddb38dde156ecc7f65"} Oct 30 03:05:26.375613 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:26.375564 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-567b98f465-tjqxp" podStartSLOduration=2.375552308 podStartE2EDuration="2.375552308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:05:26.374746654 +0000 UTC m=+69.496313221" watchObservedRunningTime="2026-10-30 03:05:26.375552308 +0000 UTC m=+69.497118874" Oct 30 03:05:27.361008 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.360947 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-77cf5b8dc6-dkclh" podUID="7317009d-02e3-4d2e-b8c8-13cf10b305e3" containerName="console" containerID="cri-o://7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e" gracePeriod=15 Oct 30 03:05:27.646789 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.646760 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:05:27.739944 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.739914 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-oauth-config\") pod \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " Oct 30 03:05:27.740073 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.739964 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-serving-cert\") pod \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " Oct 30 03:05:27.740073 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740004 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7317009d-02e3-4d2e-b8c8-13cf10b305e3-kube-api-access-dftdx\" (UniqueName: \"kubernetes.io/projected/7317009d-02e3-4d2e-b8c8-13cf10b305e3-kube-api-access-dftdx\") pod \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " Oct 30 03:05:27.740073 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740026 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-service-ca\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-service-ca\") pod \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " Oct 30 03:05:27.740073 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740044 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-oauth-serving-cert\") pod \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " Oct 30 03:05:27.740283 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740101 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-config\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-config\") pod \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " Oct 30 03:05:27.740283 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740202 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7317009d-02e3-4d2e-b8c8-13cf10b305e3-tmp\" (UniqueName: \"kubernetes.io/empty-dir/7317009d-02e3-4d2e-b8c8-13cf10b305e3-tmp\") pod \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\" (UID: \"7317009d-02e3-4d2e-b8c8-13cf10b305e3\") " Oct 30 03:05:27.740561 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740511 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7317009d-02e3-4d2e-b8c8-13cf10b305e3-tmp" pod "7317009d-02e3-4d2e-b8c8-13cf10b305e3" (UID: "7317009d-02e3-4d2e-b8c8-13cf10b305e3"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 03:05:27.740561 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740552 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-service-ca" pod "7317009d-02e3-4d2e-b8c8-13cf10b305e3" (UID: "7317009d-02e3-4d2e-b8c8-13cf10b305e3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:05:27.740722 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740539 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-oauth-serving-cert" pod "7317009d-02e3-4d2e-b8c8-13cf10b305e3" (UID: "7317009d-02e3-4d2e-b8c8-13cf10b305e3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:05:27.740797 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.740777 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-config" pod "7317009d-02e3-4d2e-b8c8-13cf10b305e3" (UID: "7317009d-02e3-4d2e-b8c8-13cf10b305e3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:05:27.742221 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.742198 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-oauth-config" pod "7317009d-02e3-4d2e-b8c8-13cf10b305e3" (UID: "7317009d-02e3-4d2e-b8c8-13cf10b305e3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:05:27.742221 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.742214 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7317009d-02e3-4d2e-b8c8-13cf10b305e3-kube-api-access-dftdx" pod "7317009d-02e3-4d2e-b8c8-13cf10b305e3" (UID: "7317009d-02e3-4d2e-b8c8-13cf10b305e3"). InnerVolumeSpecName "kube-api-access-dftdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:05:27.742361 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.742216 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-serving-cert" pod "7317009d-02e3-4d2e-b8c8-13cf10b305e3" (UID: "7317009d-02e3-4d2e-b8c8-13cf10b305e3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:05:27.841477 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.841442 4524 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-serving-cert\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:27.841477 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.841469 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dftdx\" (UniqueName: \"kubernetes.io/projected/7317009d-02e3-4d2e-b8c8-13cf10b305e3-kube-api-access-dftdx\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:27.841477 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.841479 4524 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-service-ca\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:27.841477 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.841488 4524 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-oauth-serving-cert\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:27.841747 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.841498 4524 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:27.841747 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.841506 4524 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7317009d-02e3-4d2e-b8c8-13cf10b305e3-tmp\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:27.841747 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:27.841530 4524 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7317009d-02e3-4d2e-b8c8-13cf10b305e3-console-oauth-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:28.332837 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.332800 4524 generic.go:356] "Generic (PLEG): container finished" podID="7317009d-02e3-4d2e-b8c8-13cf10b305e3" containerID="7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e" exitCode=0 Oct 30 03:05:28.333019 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.332847 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77cf5b8dc6-dkclh" event={"ID":"7317009d-02e3-4d2e-b8c8-13cf10b305e3","Type":"ContainerDied","Data":"7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e"} Oct 30 03:05:28.333019 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.332854 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77cf5b8dc6-dkclh" Oct 30 03:05:28.333019 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.332888 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77cf5b8dc6-dkclh" event={"ID":"7317009d-02e3-4d2e-b8c8-13cf10b305e3","Type":"ContainerDied","Data":"eb4c32bd7644f2d15634532077d4eb83f328fcdc36d258a4649cd7c4b72d4c12"} Oct 30 03:05:28.333019 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.332908 4524 scope.go:122] "RemoveContainer" containerID="7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e" Oct 30 03:05:28.341443 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.341423 4524 scope.go:122] "RemoveContainer" containerID="7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e" Oct 30 03:05:28.341791 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:05:28.341755 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e\": container with ID starting with 7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e not found: ID does not exist" containerID="7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e" Oct 30 03:05:28.341872 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.341800 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e"} err="failed to get container status \"7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e\": rpc error: code = NotFound desc = could not find container \"7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e\": container with ID starting with 7edd45c5ad57fa71507fa2d5234b7ce9d1e565a7a9508957a7d02971a441935e not found: ID does not exist" Oct 30 03:05:28.392782 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.392752 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77cf5b8dc6-dkclh"] Oct 30 03:05:28.407026 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:28.406998 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77cf5b8dc6-dkclh"] Oct 30 03:05:29.018820 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:29.018781 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7317009d-02e3-4d2e-b8c8-13cf10b305e3" path="/var/lib/kubelet/pods/7317009d-02e3-4d2e-b8c8-13cf10b305e3/volumes" Oct 30 03:05:35.026745 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.026713 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:35.027286 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.026766 4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:35.031373 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.031349 4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:35.360948 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.360922 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-567b98f465-tjqxp" Oct 30 03:05:35.449366 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.449334 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767bf888d-lmtl2"] Oct 30 03:05:35.508722 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.508694 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-wbbgs-f2qnw"] Oct 30 03:05:35.508912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.508878 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" podUID="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" containerName="insights-gathering" containerID="cri-o://857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8" gracePeriod=30 Oct 30 03:05:35.672255 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.672224 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-c7qqn-9vhg2"] Oct 30 03:05:35.672491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.672478 4524 cpu_manager.go:414] "containerMap: removing container" podUID="7317009d-02e3-4d2e-b8c8-13cf10b305e3" containerName="console" Oct 30 03:05:35.672491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.672490 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7317009d-02e3-4d2e-b8c8-13cf10b305e3" containerName="console" Oct 30 03:05:35.672630 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.672570 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7317009d-02e3-4d2e-b8c8-13cf10b305e3" containerName="console" Oct 30 03:05:35.680907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.680886 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.695796 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.695766 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-c7qqn-9vhg2"] Oct 30 03:05:35.771042 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.771021 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:05:35.802106 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.802074 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-trusted-ca-bundle\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.802240 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.802123 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/63340db1-08ab-4718-bc85-7a31df7048ff-archives-path\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.802240 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.802143 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-service-ca-bundle\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.802315 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.802281 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkhxk\" (UniqueName: \"kubernetes.io/projected/63340db1-08ab-4718-bc85-7a31df7048ff-kube-api-access-rkhxk\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.903262 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903179 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle\") pod \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " Oct 30 03:05:35.903262 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903231 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle\") pod \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " Oct 30 03:05:35.903262 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903261 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path\") pod \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " Oct 30 03:05:35.903549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903328 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc\" (UniqueName: \"kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc\") pod \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\" (UID: \"3608b350-8e02-4fc1-b866-3ef44f8ab2a9\") " Oct 30 03:05:35.903549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903388 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkhxk\" (UniqueName: \"kubernetes.io/projected/63340db1-08ab-4718-bc85-7a31df7048ff-kube-api-access-rkhxk\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.903549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903429 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-trusted-ca-bundle\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.903549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903486 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/63340db1-08ab-4718-bc85-7a31df7048ff-archives-path\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.903549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903510 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-service-ca-bundle\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.903856 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903787 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path" pod "3608b350-8e02-4fc1-b866-3ef44f8ab2a9" (UID: "3608b350-8e02-4fc1-b866-3ef44f8ab2a9"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 03:05:35.903856 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903812 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle" pod "3608b350-8e02-4fc1-b866-3ef44f8ab2a9" (UID: "3608b350-8e02-4fc1-b866-3ef44f8ab2a9"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:05:35.903856 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903821 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle" pod "3608b350-8e02-4fc1-b866-3ef44f8ab2a9" (UID: "3608b350-8e02-4fc1-b866-3ef44f8ab2a9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:05:35.904051 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.903949 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/63340db1-08ab-4718-bc85-7a31df7048ff-archives-path\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.904324 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.904302 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-service-ca-bundle\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.904460 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.904441 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-trusted-ca-bundle\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.905897 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.905877 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc" pod "3608b350-8e02-4fc1-b866-3ef44f8ab2a9" (UID: "3608b350-8e02-4fc1-b866-3ef44f8ab2a9"). InnerVolumeSpecName "kube-api-access-9xntc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:05:35.930820 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.930793 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkhxk\" (UniqueName: \"kubernetes.io/projected/63340db1-08ab-4718-bc85-7a31df7048ff-kube-api-access-rkhxk\") pod \"periodic-gathering-c7qqn-9vhg2\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:35.995613 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:35.995583 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:05:36.003877 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.003852 4524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-trusted-ca-bundle\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:36.003975 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.003878 4524 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-archives-path\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:36.003975 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.003894 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xntc\" (UniqueName: \"kubernetes.io/projected/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-kube-api-access-9xntc\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:36.003975 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.003906 4524 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3608b350-8e02-4fc1-b866-3ef44f8ab2a9-service-ca-bundle\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:05:36.309001 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.308973 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-c7qqn-9vhg2"] Oct 30 03:05:36.309611 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:05:36.309579 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63340db1_08ab_4718_bc85_7a31df7048ff.slice/crio-036d193902c3cce9d32e3625045896e7798e4fbf21f01576b6289a98fbebe074 WatchSource:0}: Error finding container 036d193902c3cce9d32e3625045896e7798e4fbf21f01576b6289a98fbebe074: Status 404 returned error can't find the container with id 036d193902c3cce9d32e3625045896e7798e4fbf21f01576b6289a98fbebe074 Oct 30 03:05:36.361243 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.361216 4524 generic.go:356] "Generic (PLEG): container finished" podID="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" containerID="857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8" exitCode=2 Oct 30 03:05:36.361363 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.361288 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" Oct 30 03:05:36.361363 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.361307 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" event={"ID":"3608b350-8e02-4fc1-b866-3ef44f8ab2a9","Type":"ContainerDied","Data":"857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8"} Oct 30 03:05:36.361471 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.361373 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-wbbgs-f2qnw" event={"ID":"3608b350-8e02-4fc1-b866-3ef44f8ab2a9","Type":"ContainerDied","Data":"566c550e6b2d4a9f3a6af414c165a450d69d85d32cdbed36ff344c545139a4c7"} Oct 30 03:05:36.361471 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.361405 4524 scope.go:122] "RemoveContainer" containerID="857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8" Oct 30 03:05:36.362729 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.362702 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" event={"ID":"63340db1-08ab-4718-bc85-7a31df7048ff","Type":"ContainerStarted","Data":"036d193902c3cce9d32e3625045896e7798e4fbf21f01576b6289a98fbebe074"} Oct 30 03:05:36.368727 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.368680 4524 scope.go:122] "RemoveContainer" containerID="857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8" Oct 30 03:05:36.368932 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:05:36.368910 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8\": container with ID starting with 857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8 not found: ID does not exist" containerID="857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8" Oct 30 03:05:36.368993 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.368943 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8"} err="failed to get container status \"857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8\": rpc error: code = NotFound desc = could not find container \"857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8\": container with ID starting with 857a957b57009dcd15b1d60c6ea2ad6935f9818b4658c30e243ef1e90dd67dd8 not found: ID does not exist" Oct 30 03:05:36.423719 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.423663 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-wbbgs-f2qnw"] Oct 30 03:05:36.433525 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:36.433494 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-wbbgs-f2qnw"] Oct 30 03:05:37.019580 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:37.019545 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" path="/var/lib/kubelet/pods/3608b350-8e02-4fc1-b866-3ef44f8ab2a9/volumes" Oct 30 03:05:37.368684 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:37.368646 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" event={"ID":"63340db1-08ab-4718-bc85-7a31df7048ff","Type":"ContainerStarted","Data":"7cf10b45ea29d60c0eb990c2e01c769da68383845fead514999b702838db9bf6"} Oct 30 03:05:37.416807 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:05:37.416743 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" podStartSLOduration=2.416730492 podStartE2EDuration="2.416730492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:05:37.415819948 +0000 UTC m=+80.537386514" watchObservedRunningTime="2026-10-30 03:05:37.416730492 +0000 UTC m=+80.538297058" Oct 30 03:05:49.335747 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:05:49.335704 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:06:00.468365 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.468312 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-767bf888d-lmtl2" podUID="6ce003de-7d0e-447f-adb3-a239839ecdc2" containerName="console" containerID="cri-o://02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b" gracePeriod=15 Oct 30 03:06:00.694282 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.694260 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:06:00.792079 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792009 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ce003de-7d0e-447f-adb3-a239839ecdc2-kube-api-access-r27cz\" (UniqueName: \"kubernetes.io/projected/6ce003de-7d0e-447f-adb3-a239839ecdc2-kube-api-access-r27cz\") pod \"6ce003de-7d0e-447f-adb3-a239839ecdc2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " Oct 30 03:06:00.792079 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792049 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-serving-cert\") pod \"6ce003de-7d0e-447f-adb3-a239839ecdc2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " Oct 30 03:06:00.792079 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792076 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-oauth-serving-cert\") pod \"6ce003de-7d0e-447f-adb3-a239839ecdc2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " Oct 30 03:06:00.792320 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792109 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-oauth-config\") pod \"6ce003de-7d0e-447f-adb3-a239839ecdc2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " Oct 30 03:06:00.792320 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792133 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-service-ca\") pod \"6ce003de-7d0e-447f-adb3-a239839ecdc2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " Oct 30 03:06:00.792320 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792149 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-trusted-ca-bundle\") pod \"6ce003de-7d0e-447f-adb3-a239839ecdc2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " Oct 30 03:06:00.792320 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792186 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ce003de-7d0e-447f-adb3-a239839ecdc2-tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ce003de-7d0e-447f-adb3-a239839ecdc2-tmp\") pod \"6ce003de-7d0e-447f-adb3-a239839ecdc2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " Oct 30 03:06:00.792320 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792227 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-config\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-config\") pod \"6ce003de-7d0e-447f-adb3-a239839ecdc2\" (UID: \"6ce003de-7d0e-447f-adb3-a239839ecdc2\") " Oct 30 03:06:00.792648 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792618 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ce003de-7d0e-447f-adb3-a239839ecdc2-tmp" pod "6ce003de-7d0e-447f-adb3-a239839ecdc2" (UID: "6ce003de-7d0e-447f-adb3-a239839ecdc2"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 03:06:00.792710 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792620 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-service-ca" pod "6ce003de-7d0e-447f-adb3-a239839ecdc2" (UID: "6ce003de-7d0e-447f-adb3-a239839ecdc2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:06:00.792710 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792665 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-oauth-serving-cert" pod "6ce003de-7d0e-447f-adb3-a239839ecdc2" (UID: "6ce003de-7d0e-447f-adb3-a239839ecdc2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:06:00.792710 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792687 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-config" pod "6ce003de-7d0e-447f-adb3-a239839ecdc2" (UID: "6ce003de-7d0e-447f-adb3-a239839ecdc2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:06:00.792710 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.792683 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-trusted-ca-bundle" pod "6ce003de-7d0e-447f-adb3-a239839ecdc2" (UID: "6ce003de-7d0e-447f-adb3-a239839ecdc2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:06:00.794239 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.794211 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-serving-cert" pod "6ce003de-7d0e-447f-adb3-a239839ecdc2" (UID: "6ce003de-7d0e-447f-adb3-a239839ecdc2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:06:00.794348 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.794250 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-oauth-config" pod "6ce003de-7d0e-447f-adb3-a239839ecdc2" (UID: "6ce003de-7d0e-447f-adb3-a239839ecdc2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:06:00.794348 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.794308 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ce003de-7d0e-447f-adb3-a239839ecdc2-kube-api-access-r27cz" pod "6ce003de-7d0e-447f-adb3-a239839ecdc2" (UID: "6ce003de-7d0e-447f-adb3-a239839ecdc2"). InnerVolumeSpecName "kube-api-access-r27cz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:06:00.892739 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.892707 4524 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ce003de-7d0e-447f-adb3-a239839ecdc2-tmp\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:06:00.892739 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.892734 4524 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:06:00.892912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.892747 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r27cz\" (UniqueName: \"kubernetes.io/projected/6ce003de-7d0e-447f-adb3-a239839ecdc2-kube-api-access-r27cz\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:06:00.892912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.892756 4524 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-serving-cert\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:06:00.892912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.892764 4524 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-oauth-serving-cert\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:06:00.892912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.892772 4524 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ce003de-7d0e-447f-adb3-a239839ecdc2-console-oauth-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:06:00.892912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.892780 4524 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-service-ca\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:06:00.892912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:00.892788 4524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ce003de-7d0e-447f-adb3-a239839ecdc2-trusted-ca-bundle\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:06:01.456048 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.456013 4524 generic.go:356] "Generic (PLEG): container finished" podID="6ce003de-7d0e-447f-adb3-a239839ecdc2" containerID="02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b" exitCode=0 Oct 30 03:06:01.456243 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.456062 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767bf888d-lmtl2" Oct 30 03:06:01.456243 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.456100 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767bf888d-lmtl2" event={"ID":"6ce003de-7d0e-447f-adb3-a239839ecdc2","Type":"ContainerDied","Data":"02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b"} Oct 30 03:06:01.456243 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.456150 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767bf888d-lmtl2" event={"ID":"6ce003de-7d0e-447f-adb3-a239839ecdc2","Type":"ContainerDied","Data":"edf47812b94545faad2c7231764fd078fb391943b04703f22c3c0c4ad47b8363"} Oct 30 03:06:01.456243 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.456175 4524 scope.go:122] "RemoveContainer" containerID="02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b" Oct 30 03:06:01.463625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.463608 4524 scope.go:122] "RemoveContainer" containerID="02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b" Oct 30 03:06:01.463881 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:06:01.463863 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b\": container with ID starting with 02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b not found: ID does not exist" containerID="02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b" Oct 30 03:06:01.463945 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.463893 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b"} err="failed to get container status \"02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b\": rpc error: code = NotFound desc = could not find container \"02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b\": container with ID starting with 02034dfc1d6aee164cea364abe065b2123e1f20c0e6a30839456e2522311d56b not found: ID does not exist" Oct 30 03:06:01.516502 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.516474 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767bf888d-lmtl2"] Oct 30 03:06:01.530639 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:01.530616 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-767bf888d-lmtl2"] Oct 30 03:06:03.018957 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:06:03.018927 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ce003de-7d0e-447f-adb3-a239839ecdc2" path="/var/lib/kubelet/pods/6ce003de-7d0e-447f-adb3-a239839ecdc2/volumes" Oct 30 03:06:19.335555 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:06:19.335500 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:06:49.336058 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:06:49.336024 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:07:00.646006 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:00.645971 4524 generic.go:356] "Generic (PLEG): container finished" podID="63340db1-08ab-4718-bc85-7a31df7048ff" containerID="7cf10b45ea29d60c0eb990c2e01c769da68383845fead514999b702838db9bf6" exitCode=0 Oct 30 03:07:00.646428 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:00.646049 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" event={"ID":"63340db1-08ab-4718-bc85-7a31df7048ff","Type":"ContainerDied","Data":"7cf10b45ea29d60c0eb990c2e01c769da68383845fead514999b702838db9bf6"} Oct 30 03:07:01.776338 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.776315 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:07:01.913257 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.913166 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63340db1-08ab-4718-bc85-7a31df7048ff-kube-api-access-rkhxk\" (UniqueName: \"kubernetes.io/projected/63340db1-08ab-4718-bc85-7a31df7048ff-kube-api-access-rkhxk\") pod \"63340db1-08ab-4718-bc85-7a31df7048ff\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " Oct 30 03:07:01.913257 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.913221 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-service-ca-bundle\") pod \"63340db1-08ab-4718-bc85-7a31df7048ff\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " Oct 30 03:07:01.913549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.913261 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63340db1-08ab-4718-bc85-7a31df7048ff-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/63340db1-08ab-4718-bc85-7a31df7048ff-archives-path\") pod \"63340db1-08ab-4718-bc85-7a31df7048ff\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " Oct 30 03:07:01.913549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.913299 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-trusted-ca-bundle\") pod \"63340db1-08ab-4718-bc85-7a31df7048ff\" (UID: \"63340db1-08ab-4718-bc85-7a31df7048ff\") " Oct 30 03:07:01.913659 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.913634 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63340db1-08ab-4718-bc85-7a31df7048ff-archives-path" pod "63340db1-08ab-4718-bc85-7a31df7048ff" (UID: "63340db1-08ab-4718-bc85-7a31df7048ff"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 03:07:01.913694 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.913655 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-service-ca-bundle" pod "63340db1-08ab-4718-bc85-7a31df7048ff" (UID: "63340db1-08ab-4718-bc85-7a31df7048ff"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:07:01.913732 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.913706 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-trusted-ca-bundle" pod "63340db1-08ab-4718-bc85-7a31df7048ff" (UID: "63340db1-08ab-4718-bc85-7a31df7048ff"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:07:01.915279 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:01.915260 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63340db1-08ab-4718-bc85-7a31df7048ff-kube-api-access-rkhxk" pod "63340db1-08ab-4718-bc85-7a31df7048ff" (UID: "63340db1-08ab-4718-bc85-7a31df7048ff"). InnerVolumeSpecName "kube-api-access-rkhxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:07:02.018622 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:02.015159 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkhxk\" (UniqueName: \"kubernetes.io/projected/63340db1-08ab-4718-bc85-7a31df7048ff-kube-api-access-rkhxk\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:07:02.018622 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:02.015196 4524 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-service-ca-bundle\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:07:02.018622 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:02.015218 4524 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/63340db1-08ab-4718-bc85-7a31df7048ff-archives-path\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:07:02.018622 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:02.015232 4524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63340db1-08ab-4718-bc85-7a31df7048ff-trusted-ca-bundle\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:07:02.653237 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:02.653205 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" event={"ID":"63340db1-08ab-4718-bc85-7a31df7048ff","Type":"ContainerDied","Data":"036d193902c3cce9d32e3625045896e7798e4fbf21f01576b6289a98fbebe074"} Oct 30 03:07:02.653402 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:02.653275 4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="036d193902c3cce9d32e3625045896e7798e4fbf21f01576b6289a98fbebe074" Oct 30 03:07:02.653402 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:07:02.653216 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-c7qqn-9vhg2" Oct 30 03:07:19.336184 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:07:19.336146 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:07:49.335132 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:07:49.335095 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:08:19.005154 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005120 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl"] Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005361 4524 cpu_manager.go:414] "containerMap: removing container" podUID="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" containerName="insights-gathering" Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005371 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" containerName="insights-gathering" Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005385 4524 cpu_manager.go:414] "containerMap: removing container" podUID="6ce003de-7d0e-447f-adb3-a239839ecdc2" containerName="console" Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005390 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ce003de-7d0e-447f-adb3-a239839ecdc2" containerName="console" Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005398 4524 cpu_manager.go:414] "containerMap: removing container" podUID="63340db1-08ab-4718-bc85-7a31df7048ff" containerName="insights-gathering" Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005402 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63340db1-08ab-4718-bc85-7a31df7048ff" containerName="insights-gathering" Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005447 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3608b350-8e02-4fc1-b866-3ef44f8ab2a9" containerName="insights-gathering" Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005454 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ce003de-7d0e-447f-adb3-a239839ecdc2" containerName="console" Oct 30 03:08:19.005556 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.005459 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="63340db1-08ab-4718-bc85-7a31df7048ff" containerName="insights-gathering" Oct 30 03:08:19.008068 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.008053 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.015005 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.014985 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7df6257d-ff60-4cdf-97ed-994b66015d43-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.015118 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.015025 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7df6257d-ff60-4cdf-97ed-994b66015d43-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.015118 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.015043 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rmtk\" (UniqueName: \"kubernetes.io/projected/7df6257d-ff60-4cdf-97ed-994b66015d43-kube-api-access-2rmtk\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.030039 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.030009 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl"] Oct 30 03:08:19.042876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.042855 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Oct 30 03:08:19.043023 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.042888 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Oct 30 03:08:19.043023 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.042891 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Oct 30 03:08:19.043023 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.042937 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Oct 30 03:08:19.043023 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.042967 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Oct 30 03:08:19.043189 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.043083 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-cnvfg\"" Oct 30 03:08:19.116206 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.116171 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rmtk\" (UniqueName: \"kubernetes.io/projected/7df6257d-ff60-4cdf-97ed-994b66015d43-kube-api-access-2rmtk\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.116417 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.116221 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7df6257d-ff60-4cdf-97ed-994b66015d43-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.116417 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.116259 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7df6257d-ff60-4cdf-97ed-994b66015d43-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.116605 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.116590 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7df6257d-ff60-4cdf-97ed-994b66015d43-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.118904 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.118881 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7df6257d-ff60-4cdf-97ed-994b66015d43-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.143928 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.143903 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rmtk\" (UniqueName: \"kubernetes.io/projected/7df6257d-ff60-4cdf-97ed-994b66015d43-kube-api-access-2rmtk\") pod \"keda-metrics-apiserver-86bd856cfb-7wjtl\" (UID: \"7df6257d-ff60-4cdf-97ed-994b66015d43\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.303940 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.303872 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-qbqhb"] Oct 30 03:08:19.307098 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.307082 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:19.317410 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.317381 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c598c9a0-e3ad-4f8c-8639-0dd6d393be36-certificates\") pod \"keda-admission-7b7c985c58-qbqhb\" (UID: \"c598c9a0-e3ad-4f8c-8639-0dd6d393be36\") " pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:19.317550 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.317429 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqc6b\" (UniqueName: \"kubernetes.io/projected/c598c9a0-e3ad-4f8c-8639-0dd6d393be36-kube-api-access-xqc6b\") pod \"keda-admission-7b7c985c58-qbqhb\" (UID: \"c598c9a0-e3ad-4f8c-8639-0dd6d393be36\") " pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:19.323738 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.323716 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:19.330987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.330961 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-qbqhb"] Oct 30 03:08:19.331394 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.331337 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Oct 30 03:08:19.342187 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:08:19.342156 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:08:19.417974 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.417948 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c598c9a0-e3ad-4f8c-8639-0dd6d393be36-certificates\") pod \"keda-admission-7b7c985c58-qbqhb\" (UID: \"c598c9a0-e3ad-4f8c-8639-0dd6d393be36\") " pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:19.417974 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.417985 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqc6b\" (UniqueName: \"kubernetes.io/projected/c598c9a0-e3ad-4f8c-8639-0dd6d393be36-kube-api-access-xqc6b\") pod \"keda-admission-7b7c985c58-qbqhb\" (UID: \"c598c9a0-e3ad-4f8c-8639-0dd6d393be36\") " pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:19.422131 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.420396 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c598c9a0-e3ad-4f8c-8639-0dd6d393be36-certificates\") pod \"keda-admission-7b7c985c58-qbqhb\" (UID: \"c598c9a0-e3ad-4f8c-8639-0dd6d393be36\") " pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:19.446080 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.446050 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqc6b\" (UniqueName: \"kubernetes.io/projected/c598c9a0-e3ad-4f8c-8639-0dd6d393be36-kube-api-access-xqc6b\") pod \"keda-admission-7b7c985c58-qbqhb\" (UID: \"c598c9a0-e3ad-4f8c-8639-0dd6d393be36\") " pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:19.619003 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.618976 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl"] Oct 30 03:08:19.619130 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.619043 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:19.619635 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:08:19.619613 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df6257d_ff60_4cdf_97ed_994b66015d43.slice/crio-1fc4e07f52d9245d9618b5a31f7afcf404b459dd6eb06a96e4e8482993a310fc WatchSource:0}: Error finding container 1fc4e07f52d9245d9618b5a31f7afcf404b459dd6eb06a96e4e8482993a310fc: Status 404 returned error can't find the container with id 1fc4e07f52d9245d9618b5a31f7afcf404b459dd6eb06a96e4e8482993a310fc Oct 30 03:08:19.899061 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.898981 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" event={"ID":"7df6257d-ff60-4cdf-97ed-994b66015d43","Type":"ContainerStarted","Data":"1fc4e07f52d9245d9618b5a31f7afcf404b459dd6eb06a96e4e8482993a310fc"} Oct 30 03:08:19.915326 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:19.915297 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-qbqhb"] Oct 30 03:08:19.916894 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:08:19.916861 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc598c9a0_e3ad_4f8c_8639_0dd6d393be36.slice/crio-e66d98d3fa00c75c9989060e0ee8953e4d46a097ae1d07411db8c6f06f963b84 WatchSource:0}: Error finding container e66d98d3fa00c75c9989060e0ee8953e4d46a097ae1d07411db8c6f06f963b84: Status 404 returned error can't find the container with id e66d98d3fa00c75c9989060e0ee8953e4d46a097ae1d07411db8c6f06f963b84 Oct 30 03:08:20.904642 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:20.904605 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" event={"ID":"c598c9a0-e3ad-4f8c-8639-0dd6d393be36","Type":"ContainerStarted","Data":"e66d98d3fa00c75c9989060e0ee8953e4d46a097ae1d07411db8c6f06f963b84"} Oct 30 03:08:23.917891 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:23.917855 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" event={"ID":"7df6257d-ff60-4cdf-97ed-994b66015d43","Type":"ContainerStarted","Data":"33b0caa15135376af20a675e55321fc33ab8e9e8cead2c5b183056d4abbde171"} Oct 30 03:08:23.918303 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:23.917971 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:23.919733 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:23.919707 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" event={"ID":"c598c9a0-e3ad-4f8c-8639-0dd6d393be36","Type":"ContainerStarted","Data":"45de291c69bb0e1d246ff3d6bfd4bba59e358c930d2237f8a36e90e563bf11e3"} Oct 30 03:08:23.919874 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:23.919755 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:23.966666 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:23.966617 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" podStartSLOduration=2.370123252 podStartE2EDuration="5.9666045s" totalImagesPullingTime="3.596481248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:08:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:08:23.966314562 +0000 UTC m=+247.087881131" watchObservedRunningTime="2026-10-30 03:08:23.9666045 +0000 UTC m=+247.088171044" Oct 30 03:08:24.015236 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:24.015194 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" podStartSLOduration=1.892672373 podStartE2EDuration="5.015184177s" totalImagesPullingTime="3.122511804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:08:24.014878643 +0000 UTC m=+247.136445206" watchObservedRunningTime="2026-10-30 03:08:24.015184177 +0000 UTC m=+247.136750743" Oct 30 03:08:34.927682 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:34.927653 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-7wjtl" Oct 30 03:08:44.926717 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:08:44.926687 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-7b7c985c58-qbqhb" Oct 30 03:08:49.336568 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:08:49.336535 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:09:19.336572 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:09:19.336534 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:09:46.477964 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.477932 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-78bb9c6ff8-lfmqz"] Oct 30 03:09:46.481060 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.481043 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:46.501215 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.501191 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-78bb9c6ff8-lfmqz"] Oct 30 03:09:46.538220 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.538196 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Oct 30 03:09:46.538326 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.538254 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Oct 30 03:09:46.544366 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.544352 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Oct 30 03:09:46.544552 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.544539 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-dg5b7\"" Oct 30 03:09:46.647165 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.647138 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/858da7f6-1adf-4997-8b7f-9f5c2d32ac69-data\") pod \"seaweedfs-78bb9c6ff8-lfmqz\" (UID: \"858da7f6-1adf-4997-8b7f-9f5c2d32ac69\") " pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:46.647291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.647198 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt2hr\" (UniqueName: \"kubernetes.io/projected/858da7f6-1adf-4997-8b7f-9f5c2d32ac69-kube-api-access-bt2hr\") pod \"seaweedfs-78bb9c6ff8-lfmqz\" (UID: \"858da7f6-1adf-4997-8b7f-9f5c2d32ac69\") " pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:46.748472 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.748404 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt2hr\" (UniqueName: \"kubernetes.io/projected/858da7f6-1adf-4997-8b7f-9f5c2d32ac69-kube-api-access-bt2hr\") pod \"seaweedfs-78bb9c6ff8-lfmqz\" (UID: \"858da7f6-1adf-4997-8b7f-9f5c2d32ac69\") " pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:46.748472 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.748459 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/858da7f6-1adf-4997-8b7f-9f5c2d32ac69-data\") pod \"seaweedfs-78bb9c6ff8-lfmqz\" (UID: \"858da7f6-1adf-4997-8b7f-9f5c2d32ac69\") " pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:46.748856 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.748840 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/858da7f6-1adf-4997-8b7f-9f5c2d32ac69-data\") pod \"seaweedfs-78bb9c6ff8-lfmqz\" (UID: \"858da7f6-1adf-4997-8b7f-9f5c2d32ac69\") " pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:46.776766 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.776740 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt2hr\" (UniqueName: \"kubernetes.io/projected/858da7f6-1adf-4997-8b7f-9f5c2d32ac69-kube-api-access-bt2hr\") pod \"seaweedfs-78bb9c6ff8-lfmqz\" (UID: \"858da7f6-1adf-4997-8b7f-9f5c2d32ac69\") " pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:46.792152 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:46.792130 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:47.085764 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:47.085736 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-78bb9c6ff8-lfmqz"] Oct 30 03:09:47.086178 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:09:47.086159 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod858da7f6_1adf_4997_8b7f_9f5c2d32ac69.slice/crio-1893117e6361cfe1c1cad32b8223f1c7363d0fd06be5dd2e413300ba0e5f3f72 WatchSource:0}: Error finding container 1893117e6361cfe1c1cad32b8223f1c7363d0fd06be5dd2e413300ba0e5f3f72: Status 404 returned error can't find the container with id 1893117e6361cfe1c1cad32b8223f1c7363d0fd06be5dd2e413300ba0e5f3f72 Oct 30 03:09:47.087826 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:47.087809 4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 03:09:47.196802 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:47.196770 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" event={"ID":"858da7f6-1adf-4997-8b7f-9f5c2d32ac69","Type":"ContainerStarted","Data":"1893117e6361cfe1c1cad32b8223f1c7363d0fd06be5dd2e413300ba0e5f3f72"} Oct 30 03:09:49.339616 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:09:49.339571 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:09:50.207778 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:50.207743 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" event={"ID":"858da7f6-1adf-4997-8b7f-9f5c2d32ac69","Type":"ContainerStarted","Data":"cba5dde4cd2b94723f62e7c29db9e031d93da17e5034fadba5504e7e87a28cc4"} Oct 30 03:09:50.207929 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:50.207862 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:09:50.255126 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:50.255084 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" podStartSLOduration=1.326718169 podStartE2EDuration="4.255071357s" totalImagesPullingTime="2.928353188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:09:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:09:50.254842125 +0000 UTC m=+333.376408693" watchObservedRunningTime="2026-10-30 03:09:50.255071357 +0000 UTC m=+333.376637971" Oct 30 03:09:56.213611 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:09:56.213581 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-78bb9c6ff8-lfmqz" Oct 30 03:10:19.336823 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:10:19.336788 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:10:49.336030 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:10:49.335996 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:11:10.493199 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.493166 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-56ddc4c448-zvrpd"] Oct 30 03:11:10.496693 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.496675 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:10.517664 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.517640 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-56ddc4c448-zvrpd"] Oct 30 03:11:10.530811 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.530793 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-c6lp9\"" Oct 30 03:11:10.530907 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.530840 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Oct 30 03:11:10.546729 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.546706 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c0ba131-95e2-42c6-807a-01c398a5efe5-tls-certs\") pod \"model-serving-api-56ddc4c448-zvrpd\" (UID: \"9c0ba131-95e2-42c6-807a-01c398a5efe5\") " pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:10.546808 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.546737 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5vdx\" (UniqueName: \"kubernetes.io/projected/9c0ba131-95e2-42c6-807a-01c398a5efe5-kube-api-access-d5vdx\") pod \"model-serving-api-56ddc4c448-zvrpd\" (UID: \"9c0ba131-95e2-42c6-807a-01c398a5efe5\") " pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:10.549224 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.549204 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-9c8nz"] Oct 30 03:11:10.552189 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.552176 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.577342 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.577323 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-9c8nz"] Oct 30 03:11:10.621956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.621937 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Oct 30 03:11:10.622105 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.622089 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-snrm8\"" Oct 30 03:11:10.622183 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.622101 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\"" Oct 30 03:11:10.647417 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.647392 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-cert\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.647507 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.647446 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rh8d\" (UniqueName: \"kubernetes.io/projected/b0f8a62e-edbe-40c7-a75b-f9154f13828d-kube-api-access-7rh8d\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.647581 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.647564 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c0ba131-95e2-42c6-807a-01c398a5efe5-tls-certs\") pod \"model-serving-api-56ddc4c448-zvrpd\" (UID: \"9c0ba131-95e2-42c6-807a-01c398a5efe5\") " pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:10.647618 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.647596 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5vdx\" (UniqueName: \"kubernetes.io/projected/9c0ba131-95e2-42c6-807a-01c398a5efe5-kube-api-access-d5vdx\") pod \"model-serving-api-56ddc4c448-zvrpd\" (UID: \"9c0ba131-95e2-42c6-807a-01c398a5efe5\") " pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:10.647669 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.647615 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.650020 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.649989 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c0ba131-95e2-42c6-807a-01c398a5efe5-tls-certs\") pod \"model-serving-api-56ddc4c448-zvrpd\" (UID: \"9c0ba131-95e2-42c6-807a-01c398a5efe5\") " pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:10.675181 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.675156 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5vdx\" (UniqueName: \"kubernetes.io/projected/9c0ba131-95e2-42c6-807a-01c398a5efe5-kube-api-access-d5vdx\") pod \"model-serving-api-56ddc4c448-zvrpd\" (UID: \"9c0ba131-95e2-42c6-807a-01c398a5efe5\") " pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:10.748319 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.748260 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.748319 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.748304 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-cert\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.748443 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.748349 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rh8d\" (UniqueName: \"kubernetes.io/projected/b0f8a62e-edbe-40c7-a75b-f9154f13828d-kube-api-access-7rh8d\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.748500 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:11:10.748479 4524 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Oct 30 03:11:10.748572 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:11:10.748561 4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-cert podName:b0f8a62e-edbe-40c7-a75b-f9154f13828d nodeName:}" failed. No retries permitted until 2026-10-30 03:11:11.248542795 +0000 UTC m=+414.370109355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-cert") pod "odh-model-controller-7f9f8cdc65-9c8nz" (UID: "b0f8a62e-edbe-40c7-a75b-f9154f13828d") : secret "odh-model-controller-webhook-cert" not found Oct 30 03:11:10.750461 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.750441 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.775822 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.775798 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rh8d\" (UniqueName: \"kubernetes.io/projected/b0f8a62e-edbe-40c7-a75b-f9154f13828d-kube-api-access-7rh8d\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:10.808450 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:10.808432 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:11.102441 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:11.102412 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-56ddc4c448-zvrpd"] Oct 30 03:11:11.102675 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:11:11.102652 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0ba131_95e2_42c6_807a_01c398a5efe5.slice/crio-fb088d800c0acf683c8c45d7b1bf6ce15d022c458bc5d0244ce08482d33f70fd WatchSource:0}: Error finding container fb088d800c0acf683c8c45d7b1bf6ce15d022c458bc5d0244ce08482d33f70fd: Status 404 returned error can't find the container with id fb088d800c0acf683c8c45d7b1bf6ce15d022c458bc5d0244ce08482d33f70fd Oct 30 03:11:11.251447 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:11.251412 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-cert\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:11.253748 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:11.253722 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0f8a62e-edbe-40c7-a75b-f9154f13828d-cert\") pod \"odh-model-controller-7f9f8cdc65-9c8nz\" (UID: \"b0f8a62e-edbe-40c7-a75b-f9154f13828d\") " pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:11.463504 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:11.463413 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:11.475373 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:11.475348 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-56ddc4c448-zvrpd" event={"ID":"9c0ba131-95e2-42c6-807a-01c398a5efe5","Type":"ContainerStarted","Data":"fb088d800c0acf683c8c45d7b1bf6ce15d022c458bc5d0244ce08482d33f70fd"} Oct 30 03:11:11.761785 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:11.761754 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-9c8nz"] Oct 30 03:11:11.763019 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:11:11.762987 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f8a62e_edbe_40c7_a75b_f9154f13828d.slice/crio-5c03961a733f9194e60e1e168cf02a0c7395245510dd4f2f1d66192ba39ec12a WatchSource:0}: Error finding container 5c03961a733f9194e60e1e168cf02a0c7395245510dd4f2f1d66192ba39ec12a: Status 404 returned error can't find the container with id 5c03961a733f9194e60e1e168cf02a0c7395245510dd4f2f1d66192ba39ec12a Oct 30 03:11:12.480968 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:12.480928 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" event={"ID":"b0f8a62e-edbe-40c7-a75b-f9154f13828d","Type":"ContainerStarted","Data":"5c03961a733f9194e60e1e168cf02a0c7395245510dd4f2f1d66192ba39ec12a"} Oct 30 03:11:14.490192 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:14.490150 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-56ddc4c448-zvrpd" event={"ID":"9c0ba131-95e2-42c6-807a-01c398a5efe5","Type":"ContainerStarted","Data":"388872082ef7fc03292a9ec6f8e2549d72097f18beb1a5070250d57802fc246b"} Oct 30 03:11:14.490645 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:14.490245 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:14.491630 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:14.491598 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" event={"ID":"b0f8a62e-edbe-40c7-a75b-f9154f13828d","Type":"ContainerStarted","Data":"a5b07a0a9d80701f4728e93d0658f3bff0390d6879c936ebdad3ffb25cd33d48"} Oct 30 03:11:14.491744 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:14.491729 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:14.539795 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:14.539742 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/model-serving-api-56ddc4c448-zvrpd" podStartSLOduration=2.087953969 podStartE2EDuration="4.539728783s" totalImagesPullingTime="2.451774814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:11:14.538751866 +0000 UTC m=+417.660318430" watchObservedRunningTime="2026-10-30 03:11:14.539728783 +0000 UTC m=+417.661295350" Oct 30 03:11:14.587003 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:14.586960 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" podStartSLOduration=1.95720721 podStartE2EDuration="4.586947449s" totalImagesPullingTime="2.629740239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:11:14.586945225 +0000 UTC m=+417.708511792" watchObservedRunningTime="2026-10-30 03:11:14.586947449 +0000 UTC m=+417.708514015" Oct 30 03:11:19.336922 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:11:19.336884 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:11:25.496475 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:25.496440 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7f9f8cdc65-9c8nz" Oct 30 03:11:25.498584 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:11:25.498568 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-56ddc4c448-zvrpd" Oct 30 03:11:49.336270 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:11:49.336238 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:12:19.336877 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:12:19.336837 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:12:49.338810 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:12:49.338774 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:13:06.587526 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.587493 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr"] Oct 30 03:13:06.590701 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.590686 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.612778 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.612752 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"kube-root-ca.crt\"" Oct 30 03:13:06.612901 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.612836 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-c4544-kube-rbac-proxy-sar-config\"" Oct 30 03:13:06.627009 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.626986 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"openshift-service-ca.crt\"" Oct 30 03:13:06.627120 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.626989 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-c4544-predictor-serving-cert\"" Oct 30 03:13:06.628815 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.628795 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr"] Oct 30 03:13:06.684645 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.684626 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.684757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.684654 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bktj7\" (UniqueName: \"kubernetes.io/projected/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-api-access-bktj7\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.684757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.684686 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f601b76-bec7-462c-aaf7-ca38301b83d4-proxy-tls\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.703558 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.703536 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"default-dockercfg-dtsbg\"" Oct 30 03:13:06.785280 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.785258 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bktj7\" (UniqueName: \"kubernetes.io/projected/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-api-access-bktj7\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.785369 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.785297 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f601b76-bec7-462c-aaf7-ca38301b83d4-proxy-tls\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.785369 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.785340 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.785950 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.785930 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.787675 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.787656 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f601b76-bec7-462c-aaf7-ca38301b83d4-proxy-tls\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.815335 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.815307 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bktj7\" (UniqueName: \"kubernetes.io/projected/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-api-access-bktj7\") pod \"success-200-isvc-c4544-predictor-698466ccfc-64xbr\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:06.904552 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:06.904477 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:07.203735 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:13:07.203705 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f601b76_bec7_462c_aaf7_ca38301b83d4.slice/crio-393afc8609486295276e8a9bef2cf0fa648e54a46c0254680e092293719b8977 WatchSource:0}: Error finding container 393afc8609486295276e8a9bef2cf0fa648e54a46c0254680e092293719b8977: Status 404 returned error can't find the container with id 393afc8609486295276e8a9bef2cf0fa648e54a46c0254680e092293719b8977 Oct 30 03:13:07.204381 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.204357 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr"] Oct 30 03:13:07.453772 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.453696 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd"] Oct 30 03:13:07.457956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.457940 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.477238 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.477215 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-c4544-kube-rbac-proxy-sar-config\"" Oct 30 03:13:07.479948 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.479915 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-c4544-predictor-serving-cert\"" Oct 30 03:13:07.481292 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.481270 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd"] Oct 30 03:13:07.490327 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.490303 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7frzs\" (UniqueName: \"kubernetes.io/projected/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-api-access-7frzs\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.490416 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.490333 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0b65a97-6ca0-49bf-b39c-a5f01149de39-proxy-tls\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.490416 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.490366 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.590693 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.590664 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0b65a97-6ca0-49bf-b39c-a5f01149de39-proxy-tls\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.591024 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.590708 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.591024 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.590754 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7frzs\" (UniqueName: \"kubernetes.io/projected/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-api-access-7frzs\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.591409 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.591384 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.593029 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.593014 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0b65a97-6ca0-49bf-b39c-a5f01149de39-proxy-tls\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.618129 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.618107 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7frzs\" (UniqueName: \"kubernetes.io/projected/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-api-access-7frzs\") pod \"error-404-isvc-c4544-predictor-6859c6d57b-h4tbd\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.773067 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.772995 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:07.888108 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:07.887957 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" event={"ID":"0f601b76-bec7-462c-aaf7-ca38301b83d4","Type":"ContainerStarted","Data":"393afc8609486295276e8a9bef2cf0fa648e54a46c0254680e092293719b8977"} Oct 30 03:13:08.093023 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:08.092984 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd"] Oct 30 03:13:08.899565 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:08.899525 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" event={"ID":"a0b65a97-6ca0-49bf-b39c-a5f01149de39","Type":"ContainerStarted","Data":"f76acd5fd6a5d65de7e1cc180e3e888920a87c35ea9b0de729ac8a7facfe6925"} Oct 30 03:13:19.346368 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:13:19.346324 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:13:24.990014 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:24.989979 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" event={"ID":"a0b65a97-6ca0-49bf-b39c-a5f01149de39","Type":"ContainerStarted","Data":"1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58"} Oct 30 03:13:24.993780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:24.993746 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" event={"ID":"0f601b76-bec7-462c-aaf7-ca38301b83d4","Type":"ContainerStarted","Data":"f26fc78ea623d7f44fd9ce8700d610f5aa172fe3ba4e03d6431c9796a62dfde9"} Oct 30 03:13:28.007292 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:28.007254 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" event={"ID":"0f601b76-bec7-462c-aaf7-ca38301b83d4","Type":"ContainerStarted","Data":"ebec0a82677a36eb1b868e14be05aa9bb9464dc158a2a0d13869d2e458e6938d"} Oct 30 03:13:28.007825 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:28.007410 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:28.007825 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:28.007562 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:28.008945 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:28.008914 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" event={"ID":"a0b65a97-6ca0-49bf-b39c-a5f01149de39","Type":"ContainerStarted","Data":"486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2"} Oct 30 03:13:28.009092 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:28.009064 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:28.009202 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:28.009167 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Oct 30 03:13:28.059161 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:28.059092 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podStartSLOduration=1.9885693930000001 podStartE2EDuration="22.059075075s" totalImagesPullingTime="20.070505682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:13:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:13:28.057500031 +0000 UTC m=+551.179066598" watchObservedRunningTime="2026-10-30 03:13:28.059075075 +0000 UTC m=+551.180641643" Oct 30 03:13:28.110567 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:28.110508 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podStartSLOduration=1.8089967040000001 podStartE2EDuration="21.110496351s" totalImagesPullingTime="19.301499647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:13:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:13:28.109983652 +0000 UTC m=+551.231550229" watchObservedRunningTime="2026-10-30 03:13:28.110496351 +0000 UTC m=+551.232062918" Oct 30 03:13:29.012405 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:29.012369 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:29.012840 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:29.012444 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Oct 30 03:13:29.013758 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:29.013734 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Oct 30 03:13:30.016310 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:30.016275 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Oct 30 03:13:34.016703 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:34.016676 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:13:34.017149 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:34.016984 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Oct 30 03:13:35.021496 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:35.021468 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:13:35.021915 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:35.021867 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Oct 30 03:13:44.017205 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:44.017164 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Oct 30 03:13:45.022018 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:45.021970 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Oct 30 03:13:49.339429 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:13:49.339392 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:13:54.017887 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:54.017846 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Oct 30 03:13:55.022451 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:13:55.022418 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Oct 30 03:14:04.017472 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:04.017391 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Oct 30 03:14:05.022475 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:05.022435 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused" Oct 30 03:14:14.017590 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:14.017549 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused" Oct 30 03:14:15.022749 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:15.022722 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:14:19.336363 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:14:19.336319 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:14:24.017690 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:24.017662 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:14:47.741478 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:47.741445 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd"] Oct 30 03:14:47.741972 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:47.741675 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" containerID="cri-o://1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58" gracePeriod=30 Oct 30 03:14:47.741972 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:47.741710 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kube-rbac-proxy" containerID="cri-o://486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2" gracePeriod=30 Oct 30 03:14:47.872477 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:47.872446 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr"] Oct 30 03:14:47.872708 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:47.872683 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" containerID="cri-o://f26fc78ea623d7f44fd9ce8700d610f5aa172fe3ba4e03d6431c9796a62dfde9" gracePeriod=30 Oct 30 03:14:47.872755 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:47.872717 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kube-rbac-proxy" containerID="cri-o://ebec0a82677a36eb1b868e14be05aa9bb9464dc158a2a0d13869d2e458e6938d" gracePeriod=30 Oct 30 03:14:48.294474 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:48.294441 4524 generic.go:356] "Generic (PLEG): container finished" podID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerID="486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2" exitCode=0 Oct 30 03:14:48.294661 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:48.294532 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" event={"ID":"a0b65a97-6ca0-49bf-b39c-a5f01149de39","Type":"ContainerDied","Data":"486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2"} Oct 30 03:14:48.296184 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:48.296163 4524 generic.go:356] "Generic (PLEG): container finished" podID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerID="ebec0a82677a36eb1b868e14be05aa9bb9464dc158a2a0d13869d2e458e6938d" exitCode=0 Oct 30 03:14:48.296284 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:48.296206 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" event={"ID":"0f601b76-bec7-462c-aaf7-ca38301b83d4","Type":"ContainerDied","Data":"ebec0a82677a36eb1b868e14be05aa9bb9464dc158a2a0d13869d2e458e6938d"} Oct 30 03:14:49.012581 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:49.012538 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused" Oct 30 03:14:49.338089 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:14:49.338054 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:14:50.017484 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:50.017446 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused" Oct 30 03:14:51.026944 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.026924 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:14:51.205124 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.205090 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0b65a97-6ca0-49bf-b39c-a5f01149de39-proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0b65a97-6ca0-49bf-b39c-a5f01149de39-proxy-tls\") pod \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " Oct 30 03:14:51.205285 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.205152 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-api-access-7frzs\" (UniqueName: \"kubernetes.io/projected/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-api-access-7frzs\") pod \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " Oct 30 03:14:51.205285 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.205183 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-rbac-proxy-sar-config\") pod \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\" (UID: \"a0b65a97-6ca0-49bf-b39c-a5f01149de39\") " Oct 30 03:14:51.205635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.205604 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-rbac-proxy-sar-config" pod "a0b65a97-6ca0-49bf-b39c-a5f01149de39" (UID: "a0b65a97-6ca0-49bf-b39c-a5f01149de39"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:14:51.207231 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.207210 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-api-access-7frzs" pod "a0b65a97-6ca0-49bf-b39c-a5f01149de39" (UID: "a0b65a97-6ca0-49bf-b39c-a5f01149de39"). InnerVolumeSpecName "kube-api-access-7frzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:14:51.207310 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.207244 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0b65a97-6ca0-49bf-b39c-a5f01149de39-proxy-tls" pod "a0b65a97-6ca0-49bf-b39c-a5f01149de39" (UID: "a0b65a97-6ca0-49bf-b39c-a5f01149de39"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:14:51.306097 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.306073 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0b65a97-6ca0-49bf-b39c-a5f01149de39-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:14:51.306097 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.306097 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7frzs\" (UniqueName: \"kubernetes.io/projected/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-api-access-7frzs\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:14:51.306229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.306111 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a0b65a97-6ca0-49bf-b39c-a5f01149de39-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:14:51.308245 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.308219 4524 generic.go:356] "Generic (PLEG): container finished" podID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerID="1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58" exitCode=0 Oct 30 03:14:51.308345 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.308274 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" Oct 30 03:14:51.308345 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.308283 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" event={"ID":"a0b65a97-6ca0-49bf-b39c-a5f01149de39","Type":"ContainerDied","Data":"1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58"} Oct 30 03:14:51.308345 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.308336 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd" event={"ID":"a0b65a97-6ca0-49bf-b39c-a5f01149de39","Type":"ContainerDied","Data":"f76acd5fd6a5d65de7e1cc180e3e888920a87c35ea9b0de729ac8a7facfe6925"} Oct 30 03:14:51.308491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.308364 4524 scope.go:122] "RemoveContainer" containerID="486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2" Oct 30 03:14:51.310145 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.310125 4524 generic.go:356] "Generic (PLEG): container finished" podID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerID="f26fc78ea623d7f44fd9ce8700d610f5aa172fe3ba4e03d6431c9796a62dfde9" exitCode=0 Oct 30 03:14:51.310238 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.310169 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" event={"ID":"0f601b76-bec7-462c-aaf7-ca38301b83d4","Type":"ContainerDied","Data":"f26fc78ea623d7f44fd9ce8700d610f5aa172fe3ba4e03d6431c9796a62dfde9"} Oct 30 03:14:51.316081 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.315948 4524 scope.go:122] "RemoveContainer" containerID="1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58" Oct 30 03:14:51.322808 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.322792 4524 scope.go:122] "RemoveContainer" containerID="486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2" Oct 30 03:14:51.323062 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:14:51.323043 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2\": container with ID starting with 486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2 not found: ID does not exist" containerID="486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2" Oct 30 03:14:51.323115 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.323069 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2"} err="failed to get container status \"486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2\": rpc error: code = NotFound desc = could not find container \"486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2\": container with ID starting with 486f74f59f73a04b19d76e41d9646b0f0540aa8682f2a83da660885da17052d2 not found: ID does not exist" Oct 30 03:14:51.323115 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.323085 4524 scope.go:122] "RemoveContainer" containerID="1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58" Oct 30 03:14:51.323299 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:14:51.323282 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58\": container with ID starting with 1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58 not found: ID does not exist" containerID="1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58" Oct 30 03:14:51.323352 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.323310 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58"} err="failed to get container status \"1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58\": rpc error: code = NotFound desc = could not find container \"1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58\": container with ID starting with 1ba3b92789216e76f5326e7387b491f42230ea95ff08109a2002458bd63e1a58 not found: ID does not exist" Oct 30 03:14:51.369420 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.369389 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd"] Oct 30 03:14:51.383607 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.383574 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-c4544-predictor-6859c6d57b-h4tbd"] Oct 30 03:14:51.452690 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.452667 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:14:51.608044 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.608008 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-api-access-bktj7\" (UniqueName: \"kubernetes.io/projected/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-api-access-bktj7\") pod \"0f601b76-bec7-462c-aaf7-ca38301b83d4\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " Oct 30 03:14:51.608203 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.608064 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f601b76-bec7-462c-aaf7-ca38301b83d4-proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f601b76-bec7-462c-aaf7-ca38301b83d4-proxy-tls\") pod \"0f601b76-bec7-462c-aaf7-ca38301b83d4\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " Oct 30 03:14:51.608203 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.608155 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-rbac-proxy-sar-config\") pod \"0f601b76-bec7-462c-aaf7-ca38301b83d4\" (UID: \"0f601b76-bec7-462c-aaf7-ca38301b83d4\") " Oct 30 03:14:51.608461 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.608433 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-rbac-proxy-sar-config" pod "0f601b76-bec7-462c-aaf7-ca38301b83d4" (UID: "0f601b76-bec7-462c-aaf7-ca38301b83d4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:14:51.610047 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.610025 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f601b76-bec7-462c-aaf7-ca38301b83d4-proxy-tls" pod "0f601b76-bec7-462c-aaf7-ca38301b83d4" (UID: "0f601b76-bec7-462c-aaf7-ca38301b83d4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:14:51.610107 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.610065 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-api-access-bktj7" pod "0f601b76-bec7-462c-aaf7-ca38301b83d4" (UID: "0f601b76-bec7-462c-aaf7-ca38301b83d4"). InnerVolumeSpecName "kube-api-access-bktj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:14:51.709457 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.709435 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:14:51.709457 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.709456 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bktj7\" (UniqueName: \"kubernetes.io/projected/0f601b76-bec7-462c-aaf7-ca38301b83d4-kube-api-access-bktj7\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:14:51.709591 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:51.709465 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f601b76-bec7-462c-aaf7-ca38301b83d4-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:14:52.316474 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:52.316446 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" Oct 30 03:14:52.316926 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:52.316446 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr" event={"ID":"0f601b76-bec7-462c-aaf7-ca38301b83d4","Type":"ContainerDied","Data":"393afc8609486295276e8a9bef2cf0fa648e54a46c0254680e092293719b8977"} Oct 30 03:14:52.316926 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:52.316588 4524 scope.go:122] "RemoveContainer" containerID="ebec0a82677a36eb1b868e14be05aa9bb9464dc158a2a0d13869d2e458e6938d" Oct 30 03:14:52.325508 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:52.325481 4524 scope.go:122] "RemoveContainer" containerID="f26fc78ea623d7f44fd9ce8700d610f5aa172fe3ba4e03d6431c9796a62dfde9" Oct 30 03:14:52.383797 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:52.383772 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr"] Oct 30 03:14:52.408423 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:52.408403 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-c4544-predictor-698466ccfc-64xbr"] Oct 30 03:14:53.018929 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.018897 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" path="/var/lib/kubelet/pods/0f601b76-bec7-462c-aaf7-ca38301b83d4/volumes" Oct 30 03:14:53.019308 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.019296 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" path="/var/lib/kubelet/pods/a0b65a97-6ca0-49bf-b39c-a5f01149de39/volumes" Oct 30 03:14:53.301444 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301378 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d"] Oct 30 03:14:53.301710 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301697 4524 cpu_manager.go:414] "containerMap: removing container" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" Oct 30 03:14:53.301755 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301712 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" Oct 30 03:14:53.301755 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301722 4524 cpu_manager.go:414] "containerMap: removing container" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kube-rbac-proxy" Oct 30 03:14:53.301755 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301727 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kube-rbac-proxy" Oct 30 03:14:53.301755 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301741 4524 cpu_manager.go:414] "containerMap: removing container" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" Oct 30 03:14:53.301755 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301747 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" Oct 30 03:14:53.301905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301756 4524 cpu_manager.go:414] "containerMap: removing container" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kube-rbac-proxy" Oct 30 03:14:53.301905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301763 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kube-rbac-proxy" Oct 30 03:14:53.301905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301821 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kserve-container" Oct 30 03:14:53.301905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301828 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kserve-container" Oct 30 03:14:53.301905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301835 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f601b76-bec7-462c-aaf7-ca38301b83d4" containerName="kube-rbac-proxy" Oct 30 03:14:53.301905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.301840 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0b65a97-6ca0-49bf-b39c-a5f01149de39" containerName="kube-rbac-proxy" Oct 30 03:14:53.305802 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.305787 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.326891 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.326864 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d"] Oct 30 03:14:53.340475 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.340442 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"default-dockercfg-dtsbg\"" Oct 30 03:14:53.340475 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.340469 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-ece8d-predictor-serving-cert\"" Oct 30 03:14:53.341181 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.341160 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"openshift-service-ca.crt\"" Oct 30 03:14:53.341181 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.341160 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-ece8d-kube-rbac-proxy-sar-config\"" Oct 30 03:14:53.341332 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.341214 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"kube-root-ca.crt\"" Oct 30 03:14:53.420759 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.420729 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-proxy-tls\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.420759 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.420763 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.420935 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.420852 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcrvc\" (UniqueName: \"kubernetes.io/projected/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-api-access-qcrvc\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.521401 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.521365 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcrvc\" (UniqueName: \"kubernetes.io/projected/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-api-access-qcrvc\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.521559 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.521415 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-proxy-tls\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.521559 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.521434 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.522043 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.522022 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.523600 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.523578 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-proxy-tls\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.557965 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.557942 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcrvc\" (UniqueName: \"kubernetes.io/projected/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-api-access-qcrvc\") pod \"success-200-isvc-ece8d-predictor-db7c547f-jrx9d\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.604441 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.604406 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm"] Oct 30 03:14:53.607848 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.607833 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.620184 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.620152 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:53.631301 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.631258 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-ece8d-kube-rbac-proxy-sar-config\"" Oct 30 03:14:53.631593 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.631566 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-ece8d-predictor-serving-cert\"" Oct 30 03:14:53.631951 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.631931 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm"] Oct 30 03:14:53.722986 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.722956 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ad26784-5dc3-4a08-b792-8591cd3abe25-proxy-tls\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.723137 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.723072 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.723137 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.723105 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28kr2\" (UniqueName: \"kubernetes.io/projected/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-api-access-28kr2\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.824386 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.824324 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.824386 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.824356 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28kr2\" (UniqueName: \"kubernetes.io/projected/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-api-access-28kr2\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.824597 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.824396 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ad26784-5dc3-4a08-b792-8591cd3abe25-proxy-tls\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.825057 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.825037 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.826669 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.826653 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ad26784-5dc3-4a08-b792-8591cd3abe25-proxy-tls\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.851559 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.851533 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28kr2\" (UniqueName: \"kubernetes.io/projected/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-api-access-28kr2\") pod \"error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:53.917383 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.917359 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d"] Oct 30 03:14:53.918058 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:14:53.918039 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5564e8b_7b44_4b4f_b3fe_9c03109f69e7.slice/crio-ce3063191f9f414c17a203b72a33622d24d80404113ddd8261a7fd51abf98612 WatchSource:0}: Error finding container ce3063191f9f414c17a203b72a33622d24d80404113ddd8261a7fd51abf98612: Status 404 returned error can't find the container with id ce3063191f9f414c17a203b72a33622d24d80404113ddd8261a7fd51abf98612 Oct 30 03:14:53.922130 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:53.922108 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:54.217180 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.217153 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm"] Oct 30 03:14:54.217806 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:14:54.217785 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad26784_5dc3_4a08_b792_8591cd3abe25.slice/crio-0a9a85cc5786a4b72eee5040f1bbc6c5714f0a0463c19030bf77354444c67a92 WatchSource:0}: Error finding container 0a9a85cc5786a4b72eee5040f1bbc6c5714f0a0463c19030bf77354444c67a92: Status 404 returned error can't find the container with id 0a9a85cc5786a4b72eee5040f1bbc6c5714f0a0463c19030bf77354444c67a92 Oct 30 03:14:54.325866 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.325836 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" event={"ID":"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7","Type":"ContainerStarted","Data":"c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b"} Oct 30 03:14:54.325982 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.325892 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" event={"ID":"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7","Type":"ContainerStarted","Data":"d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3"} Oct 30 03:14:54.326049 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.326007 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:54.326104 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.326063 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" event={"ID":"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7","Type":"ContainerStarted","Data":"ce3063191f9f414c17a203b72a33622d24d80404113ddd8261a7fd51abf98612"} Oct 30 03:14:54.327761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.327731 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" event={"ID":"4ad26784-5dc3-4a08-b792-8591cd3abe25","Type":"ContainerStarted","Data":"fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9"} Oct 30 03:14:54.328075 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.327780 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" event={"ID":"4ad26784-5dc3-4a08-b792-8591cd3abe25","Type":"ContainerStarted","Data":"e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82"} Oct 30 03:14:54.328075 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.327811 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" event={"ID":"4ad26784-5dc3-4a08-b792-8591cd3abe25","Type":"ContainerStarted","Data":"0a9a85cc5786a4b72eee5040f1bbc6c5714f0a0463c19030bf77354444c67a92"} Oct 30 03:14:54.328075 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.327970 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:54.328075 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.328010 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:14:54.329620 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.329592 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused" Oct 30 03:14:54.375405 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.375351 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podStartSLOduration=1.375335634 podStartE2EDuration="1.375335634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:14:54.374102764 +0000 UTC m=+637.495669332" watchObservedRunningTime="2026-10-30 03:14:54.375335634 +0000 UTC m=+637.496902200" Oct 30 03:14:54.424908 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:54.424859 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podStartSLOduration=1.4248459119999999 podStartE2EDuration="1.424845912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:14:54.423608179 +0000 UTC m=+637.545174771" watchObservedRunningTime="2026-10-30 03:14:54.424845912 +0000 UTC m=+637.546412478" Oct 30 03:14:55.332155 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:55.331775 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:14:55.332155 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:55.331923 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused" Oct 30 03:14:55.333293 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:55.333263 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Oct 30 03:14:56.336196 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:14:56.336149 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Oct 30 03:15:00.337579 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:00.337550 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:15:00.338157 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:00.338116 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused" Oct 30 03:15:01.340474 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:01.340449 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:15:01.340909 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:01.340884 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Oct 30 03:15:10.338672 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:10.338631 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused" Oct 30 03:15:11.341176 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:11.341132 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Oct 30 03:15:19.337787 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:15:19.337748 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:15:20.338927 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:20.338893 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused" Oct 30 03:15:21.341141 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:21.341105 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Oct 30 03:15:30.339040 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:30.338958 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused" Oct 30 03:15:31.341195 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:31.341150 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Oct 30 03:15:35.711986 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.711951 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt"] Oct 30 03:15:35.717541 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.717368 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.752754 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.752722 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"success-200-isvc-49218-predictor-serving-cert\"" Oct 30 03:15:35.752925 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.752799 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"success-200-isvc-49218-kube-rbac-proxy-sar-config\"" Oct 30 03:15:35.752925 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.752829 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"kube-root-ca.crt\"" Oct 30 03:15:35.752925 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.752833 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"openshift-service-ca.crt\"" Oct 30 03:15:35.752925 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.752857 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"default-dockercfg-tlgnp\"" Oct 30 03:15:35.754554 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.754531 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt"] Oct 30 03:15:35.828800 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.828755 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.828988 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.828810 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-proxy-tls\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.828988 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.828881 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8sr9\" (UniqueName: \"kubernetes.io/projected/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-api-access-n8sr9\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.930214 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.930169 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.930214 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.930219 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-proxy-tls\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.930467 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.930334 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8sr9\" (UniqueName: \"kubernetes.io/projected/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-api-access-n8sr9\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.930904 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.930881 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.932706 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.932684 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-proxy-tls\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:35.958696 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:35.958673 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8sr9\" (UniqueName: \"kubernetes.io/projected/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-api-access-n8sr9\") pod \"success-200-isvc-49218-predictor-657bd7bc46-tgqvt\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:36.034776 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.034683 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:36.118194 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.116493 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w"] Oct 30 03:15:36.120758 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.120731 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.142427 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.142398 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"error-404-isvc-49218-predictor-serving-cert\"" Oct 30 03:15:36.142594 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.142575 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w"] Oct 30 03:15:36.142659 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.142593 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"error-404-isvc-49218-kube-rbac-proxy-sar-config\"" Oct 30 03:15:36.232802 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.232773 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b50a08ac-8107-4d50-b52b-e50b386eefe9-proxy-tls\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.232989 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.232829 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.232989 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.232967 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfc6x\" (UniqueName: \"kubernetes.io/projected/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-api-access-kfc6x\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.334195 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.334164 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.334341 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.334322 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfc6x\" (UniqueName: \"kubernetes.io/projected/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-api-access-kfc6x\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.334428 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.334412 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b50a08ac-8107-4d50-b52b-e50b386eefe9-proxy-tls\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.334976 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.334949 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.335457 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:15:36.335431 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fb1e03_ec23_43ad_8c97_887a1b3bc281.slice/crio-1b6dd06795acf5cc7e3af9f4ed164e5c6cfa7aa1fc81a439c53c1613c3fa9f84 WatchSource:0}: Error finding container 1b6dd06795acf5cc7e3af9f4ed164e5c6cfa7aa1fc81a439c53c1613c3fa9f84: Status 404 returned error can't find the container with id 1b6dd06795acf5cc7e3af9f4ed164e5c6cfa7aa1fc81a439c53c1613c3fa9f84 Oct 30 03:15:36.336221 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.336197 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt"] Oct 30 03:15:36.337342 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.337316 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b50a08ac-8107-4d50-b52b-e50b386eefe9-proxy-tls\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.361762 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.361736 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfc6x\" (UniqueName: \"kubernetes.io/projected/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-api-access-kfc6x\") pod \"error-404-isvc-49218-predictor-7fb7cb44c9-g284w\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.439094 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.439060 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:36.497322 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.496417 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" event={"ID":"c6fb1e03-ec23-43ad-8c97-887a1b3bc281","Type":"ContainerStarted","Data":"7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358"} Oct 30 03:15:36.497322 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.496471 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" event={"ID":"c6fb1e03-ec23-43ad-8c97-887a1b3bc281","Type":"ContainerStarted","Data":"11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08"} Oct 30 03:15:36.497322 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.496491 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" event={"ID":"c6fb1e03-ec23-43ad-8c97-887a1b3bc281","Type":"ContainerStarted","Data":"1b6dd06795acf5cc7e3af9f4ed164e5c6cfa7aa1fc81a439c53c1613c3fa9f84"} Oct 30 03:15:36.497322 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.497241 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:36.497322 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.497278 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:36.498536 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.498325 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:15:36.546854 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.546809 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podStartSLOduration=1.546796005 podStartE2EDuration="1.546796005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:15:36.545914833 +0000 UTC m=+679.667481410" watchObservedRunningTime="2026-10-30 03:15:36.546796005 +0000 UTC m=+679.668362654" Oct 30 03:15:36.734114 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:36.734079 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w"] Oct 30 03:15:36.735044 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:15:36.735013 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50a08ac_8107_4d50_b52b_e50b386eefe9.slice/crio-342b720ae2f38be9bd0173cf0cf6ccdaae79dd6a9a55a2e81b066631d27ad185 WatchSource:0}: Error finding container 342b720ae2f38be9bd0173cf0cf6ccdaae79dd6a9a55a2e81b066631d27ad185: Status 404 returned error can't find the container with id 342b720ae2f38be9bd0173cf0cf6ccdaae79dd6a9a55a2e81b066631d27ad185 Oct 30 03:15:37.501931 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:37.501898 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" event={"ID":"b50a08ac-8107-4d50-b52b-e50b386eefe9","Type":"ContainerStarted","Data":"6051e552a642321de4bfc6824916cd8b84959799ccccba01356dccc87f719698"} Oct 30 03:15:37.502108 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:37.501959 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" event={"ID":"b50a08ac-8107-4d50-b52b-e50b386eefe9","Type":"ContainerStarted","Data":"347b6bf99040f77467d724c8a84764705e0631f4bba05baa48db783565635c6a"} Oct 30 03:15:37.502108 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:37.501982 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" event={"ID":"b50a08ac-8107-4d50-b52b-e50b386eefe9","Type":"ContainerStarted","Data":"342b720ae2f38be9bd0173cf0cf6ccdaae79dd6a9a55a2e81b066631d27ad185"} Oct 30 03:15:37.502270 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:37.502239 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:15:37.552145 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:37.552098 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podStartSLOduration=1.552085655 podStartE2EDuration="1.552085655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:15:37.551957068 +0000 UTC m=+680.673523636" watchObservedRunningTime="2026-10-30 03:15:37.552085655 +0000 UTC m=+680.673652261" Oct 30 03:15:38.505531 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:38.505472 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:15:38.506004 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:38.505543 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:38.506004 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:38.505631 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:38.507011 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:38.506987 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Oct 30 03:15:39.509327 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:39.509290 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Oct 30 03:15:40.338590 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:40.338552 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused" Oct 30 03:15:41.341818 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:41.341776 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Oct 30 03:15:43.510062 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:43.510035 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:15:43.510567 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:43.510543 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:15:44.513844 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:44.513815 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:15:44.514241 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:44.514208 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Oct 30 03:15:49.336002 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:15:49.335959 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:15:50.339317 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:50.339291 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:15:51.342233 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:51.342206 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:15:53.511324 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:53.511280 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:15:54.514759 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:15:54.514716 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Oct 30 03:16:03.511034 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:16:03.510994 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:16:04.514267 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:16:04.514232 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Oct 30 03:16:13.510904 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:16:13.510868 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:16:14.514459 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:16:14.514422 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Oct 30 03:16:19.340117 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:16:19.340082 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:16:23.511537 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:16:23.511489 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:16:24.515279 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:16:24.515242 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Oct 30 03:16:33.510686 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:16:33.510658 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:16:34.514650 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:16:34.514623 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:16:49.337854 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:16:49.337818 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:17:19.337051 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:17:19.336963 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:17:49.336698 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:17:49.336664 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:18:19.337216 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:18:19.337183 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:18:49.336859 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:18:49.336824 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:19:19.336707 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:19:19.336677 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:19:49.337733 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:19:49.337697 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:20:19.337074 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:20:19.336987 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:20:49.336269 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:20:49.336231 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:21:19.335966 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:21:19.335930 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:21:49.336970 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:21:49.336936 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:22:19.337081 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:22:19.337026 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:22:49.336101 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:22:49.336067 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:23:19.337158 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:23:19.337073 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:23:49.336179 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:23:49.336140 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:24:18.931341 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:18.931306 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm"] Oct 30 03:24:18.933772 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:18.931540 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" containerID="cri-o://e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82" gracePeriod=30 Oct 30 03:24:18.933772 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:18.931554 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kube-rbac-proxy" containerID="cri-o://fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9" gracePeriod=30 Oct 30 03:24:19.071693 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:19.071661 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d"] Oct 30 03:24:19.071913 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:19.071894 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" containerID="cri-o://d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3" gracePeriod=30 Oct 30 03:24:19.071982 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:19.071959 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kube-rbac-proxy" containerID="cri-o://c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b" gracePeriod=30 Oct 30 03:24:19.337760 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:24:19.337724 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:24:19.501013 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:19.500926 4524 generic.go:356] "Generic (PLEG): container finished" podID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerID="c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b" exitCode=0 Oct 30 03:24:19.501013 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:19.501002 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" event={"ID":"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7","Type":"ContainerDied","Data":"c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b"} Oct 30 03:24:19.502671 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:19.502652 4524 generic.go:356] "Generic (PLEG): container finished" podID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerID="fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9" exitCode=0 Oct 30 03:24:19.502771 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:19.502731 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" event={"ID":"4ad26784-5dc3-4a08-b792-8591cd3abe25","Type":"ContainerDied","Data":"fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9"} Oct 30 03:24:20.333123 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:20.333072 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused" Oct 30 03:24:20.338740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:20.338710 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused" Oct 30 03:24:21.337147 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:21.337096 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused" Oct 30 03:24:21.341340 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:21.341314 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused" Oct 30 03:24:22.282176 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.282152 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:24:22.328691 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.328658 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-api-access-28kr2\" (UniqueName: \"kubernetes.io/projected/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-api-access-28kr2\") pod \"4ad26784-5dc3-4a08-b792-8591cd3abe25\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " Oct 30 03:24:22.328830 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.328694 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ad26784-5dc3-4a08-b792-8591cd3abe25-proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ad26784-5dc3-4a08-b792-8591cd3abe25-proxy-tls\") pod \"4ad26784-5dc3-4a08-b792-8591cd3abe25\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " Oct 30 03:24:22.328830 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.328786 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-rbac-proxy-sar-config\") pod \"4ad26784-5dc3-4a08-b792-8591cd3abe25\" (UID: \"4ad26784-5dc3-4a08-b792-8591cd3abe25\") " Oct 30 03:24:22.329189 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.329163 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-rbac-proxy-sar-config" pod "4ad26784-5dc3-4a08-b792-8591cd3abe25" (UID: "4ad26784-5dc3-4a08-b792-8591cd3abe25"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:24:22.331300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.331263 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ad26784-5dc3-4a08-b792-8591cd3abe25-proxy-tls" pod "4ad26784-5dc3-4a08-b792-8591cd3abe25" (UID: "4ad26784-5dc3-4a08-b792-8591cd3abe25"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:24:22.331300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.331287 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-api-access-28kr2" pod "4ad26784-5dc3-4a08-b792-8591cd3abe25" (UID: "4ad26784-5dc3-4a08-b792-8591cd3abe25"). InnerVolumeSpecName "kube-api-access-28kr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:24:22.429846 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.429814 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:24:22.429846 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.429844 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28kr2\" (UniqueName: \"kubernetes.io/projected/4ad26784-5dc3-4a08-b792-8591cd3abe25-kube-api-access-28kr2\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:24:22.430239 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.429856 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ad26784-5dc3-4a08-b792-8591cd3abe25-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:24:22.503905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.503881 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:24:22.515195 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.515170 4524 generic.go:356] "Generic (PLEG): container finished" podID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerID="d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3" exitCode=0 Oct 30 03:24:22.515314 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.515215 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" Oct 30 03:24:22.515314 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.515242 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" event={"ID":"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7","Type":"ContainerDied","Data":"d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3"} Oct 30 03:24:22.515314 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.515304 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d" event={"ID":"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7","Type":"ContainerDied","Data":"ce3063191f9f414c17a203b72a33622d24d80404113ddd8261a7fd51abf98612"} Oct 30 03:24:22.515486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.515339 4524 scope.go:122] "RemoveContainer" containerID="c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b" Oct 30 03:24:22.516795 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.516769 4524 generic.go:356] "Generic (PLEG): container finished" podID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerID="e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82" exitCode=0 Oct 30 03:24:22.516890 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.516801 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" event={"ID":"4ad26784-5dc3-4a08-b792-8591cd3abe25","Type":"ContainerDied","Data":"e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82"} Oct 30 03:24:22.516890 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.516817 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" Oct 30 03:24:22.516890 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.516857 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm" event={"ID":"4ad26784-5dc3-4a08-b792-8591cd3abe25","Type":"ContainerDied","Data":"0a9a85cc5786a4b72eee5040f1bbc6c5714f0a0463c19030bf77354444c67a92"} Oct 30 03:24:22.523111 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.523091 4524 scope.go:122] "RemoveContainer" containerID="d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3" Oct 30 03:24:22.530270 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.530255 4524 scope.go:122] "RemoveContainer" containerID="c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b" Oct 30 03:24:22.530456 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.530437 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-proxy-tls\") pod \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " Oct 30 03:24:22.530501 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:24:22.530480 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b\": container with ID starting with c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b not found: ID does not exist" containerID="c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b" Oct 30 03:24:22.530565 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.530507 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b"} err="failed to get container status \"c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b\": rpc error: code = NotFound desc = could not find container \"c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b\": container with ID starting with c9e3d1e1d545fc320ce669eef493ece3a0404315314410f3812b65153f20a94b not found: ID does not exist" Oct 30 03:24:22.530565 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.530542 4524 scope.go:122] "RemoveContainer" containerID="d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3" Oct 30 03:24:22.530657 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.530527 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-api-access-qcrvc\" (UniqueName: \"kubernetes.io/projected/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-api-access-qcrvc\") pod \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " Oct 30 03:24:22.530710 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.530656 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-rbac-proxy-sar-config\") pod \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\" (UID: \"d5564e8b-7b44-4b4f-b3fe-9c03109f69e7\") " Oct 30 03:24:22.531033 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:24:22.531011 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3\": container with ID starting with d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3 not found: ID does not exist" containerID="d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3" Oct 30 03:24:22.531033 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.531020 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-rbac-proxy-sar-config" pod "d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" (UID: "d5564e8b-7b44-4b4f-b3fe-9c03109f69e7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:24:22.531172 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.531037 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3"} err="failed to get container status \"d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3\": rpc error: code = NotFound desc = could not find container \"d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3\": container with ID starting with d448b1312bae549e8a6e87a85a63e9d46e53f5410cdf9a2147acadd159ec56d3 not found: ID does not exist" Oct 30 03:24:22.531172 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.531052 4524 scope.go:122] "RemoveContainer" containerID="fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9" Oct 30 03:24:22.532609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.532587 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-api-access-qcrvc" pod "d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" (UID: "d5564e8b-7b44-4b4f-b3fe-9c03109f69e7"). InnerVolumeSpecName "kube-api-access-qcrvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:24:22.532708 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.532617 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-proxy-tls" pod "d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" (UID: "d5564e8b-7b44-4b4f-b3fe-9c03109f69e7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:24:22.538626 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.538591 4524 scope.go:122] "RemoveContainer" containerID="e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82" Oct 30 03:24:22.545210 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.545194 4524 scope.go:122] "RemoveContainer" containerID="fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9" Oct 30 03:24:22.545456 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:24:22.545440 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9\": container with ID starting with fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9 not found: ID does not exist" containerID="fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9" Oct 30 03:24:22.545504 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.545471 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9"} err="failed to get container status \"fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9\": rpc error: code = NotFound desc = could not find container \"fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9\": container with ID starting with fefe90d414504f8f8078a27da8c94f40b493aeef934f78a2cf062af1a4e324e9 not found: ID does not exist" Oct 30 03:24:22.545504 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.545488 4524 scope.go:122] "RemoveContainer" containerID="e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82" Oct 30 03:24:22.545724 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:24:22.545709 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82\": container with ID starting with e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82 not found: ID does not exist" containerID="e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82" Oct 30 03:24:22.545767 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.545727 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82"} err="failed to get container status \"e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82\": rpc error: code = NotFound desc = could not find container \"e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82\": container with ID starting with e5aec8a9d831df2cb20b3eb041a4548576c8367b786619bee35069fa7c127d82 not found: ID does not exist" Oct 30 03:24:22.615074 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.615042 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm"] Oct 30 03:24:22.622808 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.622781 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-ece8d-predictor-67f6456dc8-d7fnm"] Oct 30 03:24:22.632095 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.632072 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:24:22.632181 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.632094 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:24:22.632181 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.632109 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcrvc\" (UniqueName: \"kubernetes.io/projected/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7-kube-api-access-qcrvc\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:24:22.867172 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.867145 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d"] Oct 30 03:24:22.873611 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:22.873587 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-ece8d-predictor-db7c547f-jrx9d"] Oct 30 03:24:23.019797 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:23.019759 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" path="/var/lib/kubelet/pods/4ad26784-5dc3-4a08-b792-8591cd3abe25/volumes" Oct 30 03:24:23.020192 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:23.020180 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" path="/var/lib/kubelet/pods/d5564e8b-7b44-4b4f-b3fe-9c03109f69e7/volumes" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.457956 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx"] Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458613 4524 cpu_manager.go:414] "containerMap: removing container" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kube-rbac-proxy" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458633 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kube-rbac-proxy" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458650 4524 cpu_manager.go:414] "containerMap: removing container" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458658 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458703 4524 cpu_manager.go:414] "containerMap: removing container" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kube-rbac-proxy" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458712 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kube-rbac-proxy" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458745 4524 cpu_manager.go:414] "containerMap: removing container" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458753 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458883 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kube-rbac-proxy" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458904 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ad26784-5dc3-4a08-b792-8591cd3abe25" containerName="kserve-container" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458916 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kube-rbac-proxy" Oct 30 03:24:24.460609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.458933 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5564e8b-7b44-4b4f-b3fe-9c03109f69e7" containerName="kserve-container" Oct 30 03:24:24.463920 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.463898 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.480926 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.480906 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx"] Oct 30 03:24:24.494228 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.494207 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"kube-root-ca.crt\"" Oct 30 03:24:24.494337 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.494234 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"openshift-service-ca.crt\"" Oct 30 03:24:24.494337 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.494272 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-5eb02-kube-rbac-proxy-sar-config\"" Oct 30 03:24:24.496286 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.496268 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"default-dockercfg-dtsbg\"" Oct 30 03:24:24.496286 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.496280 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-5eb02-predictor-serving-cert\"" Oct 30 03:24:24.547796 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.547763 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmcrn\" (UniqueName: \"kubernetes.io/projected/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-api-access-cmcrn\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.547939 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.547840 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee96b447-e45f-48eb-9df0-2f4f4bb46039-proxy-tls\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.547939 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.547896 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.648527 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.648488 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee96b447-e45f-48eb-9df0-2f4f4bb46039-proxy-tls\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.648706 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.648552 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.648706 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.648604 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmcrn\" (UniqueName: \"kubernetes.io/projected/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-api-access-cmcrn\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.649327 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.649298 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.651060 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.651042 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee96b447-e45f-48eb-9df0-2f4f4bb46039-proxy-tls\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.676931 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.676907 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmcrn\" (UniqueName: \"kubernetes.io/projected/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-api-access-cmcrn\") pod \"success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.746334 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.746272 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p"] Oct 30 03:24:24.750002 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.749978 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.771427 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.771406 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p"] Oct 30 03:24:24.773919 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.773897 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-5eb02-predictor-serving-cert\"" Oct 30 03:24:24.774122 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.774106 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-5eb02-kube-rbac-proxy-sar-config\"" Oct 30 03:24:24.778291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.778273 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:24.850173 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.850146 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.850306 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.850180 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbrxd\" (UniqueName: \"kubernetes.io/projected/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-api-access-zbrxd\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.850306 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.850209 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-proxy-tls\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.951491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.951448 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.951491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.951495 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbrxd\" (UniqueName: \"kubernetes.io/projected/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-api-access-zbrxd\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.951765 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.951553 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-proxy-tls\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.952150 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.952128 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.954064 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.954036 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-proxy-tls\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:24.979282 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:24.979258 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbrxd\" (UniqueName: \"kubernetes.io/projected/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-api-access-zbrxd\") pod \"error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:25.064463 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.064440 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:25.076318 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.076265 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx"] Oct 30 03:24:25.076432 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:24:25.076412 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee96b447_e45f_48eb_9df0_2f4f4bb46039.slice/crio-fa88c8ccb07a146f0e0252e63ea1a83cba9ac9069f94eaa69216a5405c94798e WatchSource:0}: Error finding container fa88c8ccb07a146f0e0252e63ea1a83cba9ac9069f94eaa69216a5405c94798e: Status 404 returned error can't find the container with id fa88c8ccb07a146f0e0252e63ea1a83cba9ac9069f94eaa69216a5405c94798e Oct 30 03:24:25.365365 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:24:25.365337 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f3f445_8ef6_4d13_b4e5_73939108c7e9.slice/crio-953e9c57454b425d690ec87ba9df382912886a8387eb7cf6e13859f8a5c6d6f3 WatchSource:0}: Error finding container 953e9c57454b425d690ec87ba9df382912886a8387eb7cf6e13859f8a5c6d6f3: Status 404 returned error can't find the container with id 953e9c57454b425d690ec87ba9df382912886a8387eb7cf6e13859f8a5c6d6f3 Oct 30 03:24:25.365490 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.365451 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p"] Oct 30 03:24:25.533619 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.533579 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" event={"ID":"f0f3f445-8ef6-4d13-b4e5-73939108c7e9","Type":"ContainerStarted","Data":"abf1a107734651b081eefc70d32edd5b61f9a300a64226021b4edf542316a2b1"} Oct 30 03:24:25.534213 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.533656 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" event={"ID":"f0f3f445-8ef6-4d13-b4e5-73939108c7e9","Type":"ContainerStarted","Data":"33fdb4b24adf78945db58aa49b6ee9c855f8e18e60bc716553f3926191e51fe9"} Oct 30 03:24:25.534213 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.533690 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" event={"ID":"f0f3f445-8ef6-4d13-b4e5-73939108c7e9","Type":"ContainerStarted","Data":"953e9c57454b425d690ec87ba9df382912886a8387eb7cf6e13859f8a5c6d6f3"} Oct 30 03:24:25.534213 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.533713 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:25.536166 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.536133 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" event={"ID":"ee96b447-e45f-48eb-9df0-2f4f4bb46039","Type":"ContainerStarted","Data":"9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5"} Oct 30 03:24:25.536268 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.536197 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" event={"ID":"ee96b447-e45f-48eb-9df0-2f4f4bb46039","Type":"ContainerStarted","Data":"13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644"} Oct 30 03:24:25.536268 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.536228 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" event={"ID":"ee96b447-e45f-48eb-9df0-2f4f4bb46039","Type":"ContainerStarted","Data":"fa88c8ccb07a146f0e0252e63ea1a83cba9ac9069f94eaa69216a5405c94798e"} Oct 30 03:24:25.536566 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.536545 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:25.536656 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.536589 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:25.537973 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.537947 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Oct 30 03:24:25.583163 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.583121 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podStartSLOduration=1.583109627 podStartE2EDuration="1.583109627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:24:25.582633779 +0000 UTC m=+1208.704200353" watchObservedRunningTime="2026-10-30 03:24:25.583109627 +0000 UTC m=+1208.704676194" Oct 30 03:24:25.632622 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:25.632503 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podStartSLOduration=1.6324850990000002 podStartE2EDuration="1.632485099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:24:25.632024 +0000 UTC m=+1208.753590583" watchObservedRunningTime="2026-10-30 03:24:25.632485099 +0000 UTC m=+1208.754051666" Oct 30 03:24:26.540274 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:26.540242 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:26.540688 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:26.540293 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Oct 30 03:24:26.541639 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:26.541613 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Oct 30 03:24:27.543577 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:27.543534 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Oct 30 03:24:31.544645 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:31.544612 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:24:31.545011 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:31.544979 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Oct 30 03:24:32.547730 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:32.547702 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:24:32.548129 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:32.548105 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Oct 30 03:24:41.546048 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:41.546010 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Oct 30 03:24:42.548092 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:42.548051 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Oct 30 03:24:49.338926 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:24:49.338890 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:24:51.545294 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:51.545254 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Oct 30 03:24:52.548458 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:24:52.548423 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Oct 30 03:25:01.545699 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:01.545658 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Oct 30 03:25:01.795934 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:01.795857 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w"] Oct 30 03:25:01.796126 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:01.796078 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" containerID="cri-o://347b6bf99040f77467d724c8a84764705e0631f4bba05baa48db783565635c6a" gracePeriod=30 Oct 30 03:25:01.796201 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:01.796113 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kube-rbac-proxy" containerID="cri-o://6051e552a642321de4bfc6824916cd8b84959799ccccba01356dccc87f719698" gracePeriod=30 Oct 30 03:25:01.901501 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:01.901470 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt"] Oct 30 03:25:01.901737 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:01.901702 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" containerID="cri-o://11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08" gracePeriod=30 Oct 30 03:25:01.901864 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:01.901778 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kube-rbac-proxy" containerID="cri-o://7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358" gracePeriod=30 Oct 30 03:25:02.548131 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:02.548096 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Oct 30 03:25:02.681726 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:02.681694 4524 generic.go:356] "Generic (PLEG): container finished" podID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerID="6051e552a642321de4bfc6824916cd8b84959799ccccba01356dccc87f719698" exitCode=0 Oct 30 03:25:02.681906 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:02.681765 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" event={"ID":"b50a08ac-8107-4d50-b52b-e50b386eefe9","Type":"ContainerDied","Data":"6051e552a642321de4bfc6824916cd8b84959799ccccba01356dccc87f719698"} Oct 30 03:25:02.683503 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:02.683478 4524 generic.go:356] "Generic (PLEG): container finished" podID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerID="7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358" exitCode=0 Oct 30 03:25:02.683618 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:02.683545 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" event={"ID":"c6fb1e03-ec23-43ad-8c97-887a1b3bc281","Type":"ContainerDied","Data":"7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358"} Oct 30 03:25:03.506387 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:03.506346 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused" Oct 30 03:25:03.510707 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:03.510680 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused" Oct 30 03:25:04.510490 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:04.510444 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused" Oct 30 03:25:04.514839 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:04.514815 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Oct 30 03:25:05.433732 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.433699 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:25:05.580737 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.580708 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-proxy-tls\") pod \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " Oct 30 03:25:05.581096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.580742 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-api-access-n8sr9\" (UniqueName: \"kubernetes.io/projected/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-api-access-n8sr9\") pod \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " Oct 30 03:25:05.581096 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.580771 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-rbac-proxy-sar-config\") pod \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\" (UID: \"c6fb1e03-ec23-43ad-8c97-887a1b3bc281\") " Oct 30 03:25:05.581182 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.581122 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-rbac-proxy-sar-config" pod "c6fb1e03-ec23-43ad-8c97-887a1b3bc281" (UID: "c6fb1e03-ec23-43ad-8c97-887a1b3bc281"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:25:05.582933 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.582911 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-api-access-n8sr9" pod "c6fb1e03-ec23-43ad-8c97-887a1b3bc281" (UID: "c6fb1e03-ec23-43ad-8c97-887a1b3bc281"). InnerVolumeSpecName "kube-api-access-n8sr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:25:05.582998 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.582928 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-proxy-tls" pod "c6fb1e03-ec23-43ad-8c97-887a1b3bc281" (UID: "c6fb1e03-ec23-43ad-8c97-887a1b3bc281"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:25:05.682197 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.682168 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:05.682197 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.682196 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8sr9\" (UniqueName: \"kubernetes.io/projected/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-api-access-n8sr9\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:05.682357 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.682210 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6fb1e03-ec23-43ad-8c97-887a1b3bc281-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:05.697138 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.697109 4524 generic.go:356] "Generic (PLEG): container finished" podID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerID="11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08" exitCode=0 Oct 30 03:25:05.697291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.697182 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" Oct 30 03:25:05.697291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.697191 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" event={"ID":"c6fb1e03-ec23-43ad-8c97-887a1b3bc281","Type":"ContainerDied","Data":"11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08"} Oct 30 03:25:05.697291 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.697251 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt" event={"ID":"c6fb1e03-ec23-43ad-8c97-887a1b3bc281","Type":"ContainerDied","Data":"1b6dd06795acf5cc7e3af9f4ed164e5c6cfa7aa1fc81a439c53c1613c3fa9f84"} Oct 30 03:25:05.697448 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.697290 4524 scope.go:122] "RemoveContainer" containerID="7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358" Oct 30 03:25:05.699053 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.699032 4524 generic.go:356] "Generic (PLEG): container finished" podID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerID="347b6bf99040f77467d724c8a84764705e0631f4bba05baa48db783565635c6a" exitCode=0 Oct 30 03:25:05.699156 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.699062 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" event={"ID":"b50a08ac-8107-4d50-b52b-e50b386eefe9","Type":"ContainerDied","Data":"347b6bf99040f77467d724c8a84764705e0631f4bba05baa48db783565635c6a"} Oct 30 03:25:05.706278 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.706248 4524 scope.go:122] "RemoveContainer" containerID="11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08" Oct 30 03:25:05.713868 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.713852 4524 scope.go:122] "RemoveContainer" containerID="7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358" Oct 30 03:25:05.714149 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:25:05.714127 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358\": container with ID starting with 7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358 not found: ID does not exist" containerID="7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358" Oct 30 03:25:05.714241 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.714156 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358"} err="failed to get container status \"7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358\": rpc error: code = NotFound desc = could not find container \"7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358\": container with ID starting with 7e6aec56beff9ac4f0bf8aa6bd1bddb04bf1222168ac19345a1a2597eb86a358 not found: ID does not exist" Oct 30 03:25:05.714241 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.714175 4524 scope.go:122] "RemoveContainer" containerID="11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08" Oct 30 03:25:05.714435 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:25:05.714418 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08\": container with ID starting with 11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08 not found: ID does not exist" containerID="11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08" Oct 30 03:25:05.714482 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.714437 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08"} err="failed to get container status \"11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08\": rpc error: code = NotFound desc = could not find container \"11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08\": container with ID starting with 11018aad5d53530a1a48e4de35fee8e1971f3c06bb639be0dd7a9b9f95459d08 not found: ID does not exist" Oct 30 03:25:05.727688 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.727662 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:25:05.760044 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.760016 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt"] Oct 30 03:25:05.766800 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.766776 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-49218-predictor-657bd7bc46-tgqvt"] Oct 30 03:25:05.883305 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.883270 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b50a08ac-8107-4d50-b52b-e50b386eefe9-proxy-tls\" (UniqueName: \"kubernetes.io/secret/b50a08ac-8107-4d50-b52b-e50b386eefe9-proxy-tls\") pod \"b50a08ac-8107-4d50-b52b-e50b386eefe9\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " Oct 30 03:25:05.883489 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.883361 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-rbac-proxy-sar-config\") pod \"b50a08ac-8107-4d50-b52b-e50b386eefe9\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " Oct 30 03:25:05.883489 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.883412 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-api-access-kfc6x\" (UniqueName: \"kubernetes.io/projected/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-api-access-kfc6x\") pod \"b50a08ac-8107-4d50-b52b-e50b386eefe9\" (UID: \"b50a08ac-8107-4d50-b52b-e50b386eefe9\") " Oct 30 03:25:05.883790 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.883763 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-rbac-proxy-sar-config" pod "b50a08ac-8107-4d50-b52b-e50b386eefe9" (UID: "b50a08ac-8107-4d50-b52b-e50b386eefe9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:25:05.885358 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.885331 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b50a08ac-8107-4d50-b52b-e50b386eefe9-proxy-tls" pod "b50a08ac-8107-4d50-b52b-e50b386eefe9" (UID: "b50a08ac-8107-4d50-b52b-e50b386eefe9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:25:05.885432 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.885368 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-api-access-kfc6x" pod "b50a08ac-8107-4d50-b52b-e50b386eefe9" (UID: "b50a08ac-8107-4d50-b52b-e50b386eefe9"). InnerVolumeSpecName "kube-api-access-kfc6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:25:05.984233 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.984200 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:05.984233 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.984228 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfc6x\" (UniqueName: \"kubernetes.io/projected/b50a08ac-8107-4d50-b52b-e50b386eefe9-kube-api-access-kfc6x\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:05.984233 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:05.984239 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b50a08ac-8107-4d50-b52b-e50b386eefe9-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:06.705256 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:06.705215 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" event={"ID":"b50a08ac-8107-4d50-b52b-e50b386eefe9","Type":"ContainerDied","Data":"342b720ae2f38be9bd0173cf0cf6ccdaae79dd6a9a55a2e81b066631d27ad185"} Oct 30 03:25:06.705684 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:06.705278 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w" Oct 30 03:25:06.705684 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:06.705332 4524 scope.go:122] "RemoveContainer" containerID="6051e552a642321de4bfc6824916cd8b84959799ccccba01356dccc87f719698" Oct 30 03:25:06.713315 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:06.713295 4524 scope.go:122] "RemoveContainer" containerID="347b6bf99040f77467d724c8a84764705e0631f4bba05baa48db783565635c6a" Oct 30 03:25:06.768624 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:06.768601 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w"] Oct 30 03:25:06.782883 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:06.782859 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-49218-predictor-7fb7cb44c9-g284w"] Oct 30 03:25:07.021005 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:07.020930 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" path="/var/lib/kubelet/pods/b50a08ac-8107-4d50-b52b-e50b386eefe9/volumes" Oct 30 03:25:07.021341 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:07.021328 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" path="/var/lib/kubelet/pods/c6fb1e03-ec23-43ad-8c97-887a1b3bc281/volumes" Oct 30 03:25:09.304340 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304307 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc"] Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304638 4524 cpu_manager.go:414] "containerMap: removing container" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304651 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304674 4524 cpu_manager.go:414] "containerMap: removing container" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304679 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304688 4524 cpu_manager.go:414] "containerMap: removing container" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kube-rbac-proxy" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304694 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kube-rbac-proxy" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304699 4524 cpu_manager.go:414] "containerMap: removing container" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kube-rbac-proxy" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304704 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kube-rbac-proxy" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304753 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kserve-container" Oct 30 03:25:09.304761 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304760 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6fb1e03-ec23-43ad-8c97-887a1b3bc281" containerName="kube-rbac-proxy" Oct 30 03:25:09.305083 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304768 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kube-rbac-proxy" Oct 30 03:25:09.305083 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.304775 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b50a08ac-8107-4d50-b52b-e50b386eefe9" containerName="kserve-container" Oct 30 03:25:09.307718 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.307699 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.329939 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.329914 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc"] Oct 30 03:25:09.342541 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.342506 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"default-dockercfg-tlgnp\"" Oct 30 03:25:09.342648 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.342591 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"kube-root-ca.crt\"" Oct 30 03:25:09.342648 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.342591 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"success-200-isvc-a3800-predictor-serving-cert\"" Oct 30 03:25:09.342648 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.342610 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"openshift-service-ca.crt\"" Oct 30 03:25:09.342648 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.342621 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"success-200-isvc-a3800-kube-rbac-proxy-sar-config\"" Oct 30 03:25:09.410172 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.410145 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn76g\" (UniqueName: \"kubernetes.io/projected/c718652c-c295-4851-816b-6466076e59b6-kube-api-access-kn76g\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.410274 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.410192 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c718652c-c295-4851-816b-6466076e59b6-proxy-tls\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.410329 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.410291 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c718652c-c295-4851-816b-6466076e59b6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.511675 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.511643 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c718652c-c295-4851-816b-6466076e59b6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.511831 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.511691 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn76g\" (UniqueName: \"kubernetes.io/projected/c718652c-c295-4851-816b-6466076e59b6-kube-api-access-kn76g\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.511831 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.511726 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c718652c-c295-4851-816b-6466076e59b6-proxy-tls\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.512340 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.512316 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c718652c-c295-4851-816b-6466076e59b6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.514101 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.514083 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c718652c-c295-4851-816b-6466076e59b6-proxy-tls\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.539719 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.539695 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn76g\" (UniqueName: \"kubernetes.io/projected/c718652c-c295-4851-816b-6466076e59b6-kube-api-access-kn76g\") pod \"success-200-isvc-a3800-predictor-cc47c895d-6r8mc\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.622435 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.622409 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:09.924300 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:25:09.924264 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc718652c_c295_4851_816b_6466076e59b6.slice/crio-076f75e7f60be1ec24fdeca4afe76a2f55225b9df70b94b12e92ae67456f9c33 WatchSource:0}: Error finding container 076f75e7f60be1ec24fdeca4afe76a2f55225b9df70b94b12e92ae67456f9c33: Status 404 returned error can't find the container with id 076f75e7f60be1ec24fdeca4afe76a2f55225b9df70b94b12e92ae67456f9c33 Oct 30 03:25:09.924459 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.924440 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc"] Oct 30 03:25:09.985177 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.985152 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2"] Oct 30 03:25:09.988683 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:09.988668 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.009595 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.009548 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"error-404-isvc-a3800-predictor-serving-cert\"" Oct 30 03:25:10.010357 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.010260 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2"] Oct 30 03:25:10.010464 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.010447 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"error-404-isvc-a3800-kube-rbac-proxy-sar-config\"" Oct 30 03:25:10.116462 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.116431 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-proxy-tls\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.116612 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.116503 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.116612 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.116586 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tkc8\" (UniqueName: \"kubernetes.io/projected/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-api-access-9tkc8\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.217508 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.217429 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.217508 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.217476 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tkc8\" (UniqueName: \"kubernetes.io/projected/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-api-access-9tkc8\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.217732 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.217511 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-proxy-tls\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.218163 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.218132 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.219846 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.219824 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-proxy-tls\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.245602 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.245571 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tkc8\" (UniqueName: \"kubernetes.io/projected/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-api-access-9tkc8\") pod \"error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.304544 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.304492 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.603166 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:25:10.603133 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c00590c_fd7b_45ca_92d4_fcbdff7fc8fe.slice/crio-cf32fb01983ee6581e01b83853d65aac2629cc5e317630fb8ed0d29709290868 WatchSource:0}: Error finding container cf32fb01983ee6581e01b83853d65aac2629cc5e317630fb8ed0d29709290868: Status 404 returned error can't find the container with id cf32fb01983ee6581e01b83853d65aac2629cc5e317630fb8ed0d29709290868 Oct 30 03:25:10.603956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.603925 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2"] Oct 30 03:25:10.723416 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.723391 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" event={"ID":"c718652c-c295-4851-816b-6466076e59b6","Type":"ContainerStarted","Data":"9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04"} Oct 30 03:25:10.723508 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.723451 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" event={"ID":"c718652c-c295-4851-816b-6466076e59b6","Type":"ContainerStarted","Data":"e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082"} Oct 30 03:25:10.723508 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.723472 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" event={"ID":"c718652c-c295-4851-816b-6466076e59b6","Type":"ContainerStarted","Data":"076f75e7f60be1ec24fdeca4afe76a2f55225b9df70b94b12e92ae67456f9c33"} Oct 30 03:25:10.723508 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.723500 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:10.723657 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.723542 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:10.725246 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.725224 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused" Oct 30 03:25:10.725339 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.725249 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" event={"ID":"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe","Type":"ContainerStarted","Data":"f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3"} Oct 30 03:25:10.725339 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.725299 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" event={"ID":"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe","Type":"ContainerStarted","Data":"93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37"} Oct 30 03:25:10.725339 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.725329 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" event={"ID":"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe","Type":"ContainerStarted","Data":"cf32fb01983ee6581e01b83853d65aac2629cc5e317630fb8ed0d29709290868"} Oct 30 03:25:10.725446 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.725373 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:10.773826 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.773779 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podStartSLOduration=1.7737659620000001 podStartE2EDuration="1.773765962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:25:10.773576726 +0000 UTC m=+1253.895143292" watchObservedRunningTime="2026-10-30 03:25:10.773765962 +0000 UTC m=+1253.895332529" Oct 30 03:25:10.824158 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:10.824114 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podStartSLOduration=1.8241017529999999 podStartE2EDuration="1.824101753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:25:10.823445828 +0000 UTC m=+1253.945012397" watchObservedRunningTime="2026-10-30 03:25:10.824101753 +0000 UTC m=+1253.945668320" Oct 30 03:25:11.545389 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:11.545345 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Oct 30 03:25:11.731289 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:11.731252 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:11.732148 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:11.731988 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused" Oct 30 03:25:11.733045 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:11.733016 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused" Oct 30 03:25:12.548792 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:12.548758 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Oct 30 03:25:12.736924 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:12.736888 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused" Oct 30 03:25:16.736230 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:16.736196 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:25:16.736738 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:16.736712 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused" Oct 30 03:25:17.741964 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:17.741936 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:25:17.742391 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:17.742365 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused" Oct 30 03:25:19.337933 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:25:19.337893 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:25:21.545655 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:21.545626 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:25:22.548628 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:22.548598 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:25:26.737391 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:26.737354 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused" Oct 30 03:25:27.743077 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:27.743033 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused" Oct 30 03:25:36.736692 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:36.736652 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused" Oct 30 03:25:37.742724 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:37.742683 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused" Oct 30 03:25:45.649080 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:45.649041 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p"] Oct 30 03:25:45.649457 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:45.649266 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" containerID="cri-o://33fdb4b24adf78945db58aa49b6ee9c855f8e18e60bc716553f3926191e51fe9" gracePeriod=30 Oct 30 03:25:45.649457 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:45.649307 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kube-rbac-proxy" containerID="cri-o://abf1a107734651b081eefc70d32edd5b61f9a300a64226021b4edf542316a2b1" gracePeriod=30 Oct 30 03:25:45.753222 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:45.753192 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx"] Oct 30 03:25:45.753491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:45.753447 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" containerID="cri-o://13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644" gracePeriod=30 Oct 30 03:25:45.753618 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:45.753551 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kube-rbac-proxy" containerID="cri-o://9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5" gracePeriod=30 Oct 30 03:25:45.864379 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:45.864348 4524 generic.go:356] "Generic (PLEG): container finished" podID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerID="abf1a107734651b081eefc70d32edd5b61f9a300a64226021b4edf542316a2b1" exitCode=0 Oct 30 03:25:45.864536 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:45.864392 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" event={"ID":"f0f3f445-8ef6-4d13-b4e5-73939108c7e9","Type":"ContainerDied","Data":"abf1a107734651b081eefc70d32edd5b61f9a300a64226021b4edf542316a2b1"} Oct 30 03:25:46.541405 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:46.541361 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused" Oct 30 03:25:46.737076 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:46.737022 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused" Oct 30 03:25:46.870627 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:46.870596 4524 generic.go:356] "Generic (PLEG): container finished" podID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerID="9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5" exitCode=0 Oct 30 03:25:46.870787 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:46.870662 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" event={"ID":"ee96b447-e45f-48eb-9df0-2f4f4bb46039","Type":"ContainerDied","Data":"9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5"} Oct 30 03:25:47.544630 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:47.544580 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.32:8643/healthz\": dial tcp 10.133.0.32:8643: connect: connection refused" Oct 30 03:25:47.743317 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:47.743277 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused" Oct 30 03:25:49.287660 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.287637 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:25:49.321763 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.321735 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-api-access-cmcrn\" (UniqueName: \"kubernetes.io/projected/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-api-access-cmcrn\") pod \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " Oct 30 03:25:49.321927 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.321798 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee96b447-e45f-48eb-9df0-2f4f4bb46039-proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee96b447-e45f-48eb-9df0-2f4f4bb46039-proxy-tls\") pod \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " Oct 30 03:25:49.321927 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.321841 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-rbac-proxy-sar-config\") pod \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\" (UID: \"ee96b447-e45f-48eb-9df0-2f4f4bb46039\") " Oct 30 03:25:49.322238 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.322214 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-rbac-proxy-sar-config" pod "ee96b447-e45f-48eb-9df0-2f4f4bb46039" (UID: "ee96b447-e45f-48eb-9df0-2f4f4bb46039"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:25:49.323735 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.323711 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-api-access-cmcrn" pod "ee96b447-e45f-48eb-9df0-2f4f4bb46039" (UID: "ee96b447-e45f-48eb-9df0-2f4f4bb46039"). InnerVolumeSpecName "kube-api-access-cmcrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:25:49.323882 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.323859 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee96b447-e45f-48eb-9df0-2f4f4bb46039-proxy-tls" pod "ee96b447-e45f-48eb-9df0-2f4f4bb46039" (UID: "ee96b447-e45f-48eb-9df0-2f4f4bb46039"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:25:49.339586 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:25:49.339478 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:25:49.423405 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.423312 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmcrn\" (UniqueName: \"kubernetes.io/projected/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-api-access-cmcrn\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:49.423405 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.423360 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee96b447-e45f-48eb-9df0-2f4f4bb46039-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:49.423405 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.423380 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee96b447-e45f-48eb-9df0-2f4f4bb46039-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:49.883675 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.883641 4524 generic.go:356] "Generic (PLEG): container finished" podID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerID="13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644" exitCode=0 Oct 30 03:25:49.883823 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.883708 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" Oct 30 03:25:49.883823 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.883719 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" event={"ID":"ee96b447-e45f-48eb-9df0-2f4f4bb46039","Type":"ContainerDied","Data":"13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644"} Oct 30 03:25:49.883823 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.883781 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx" event={"ID":"ee96b447-e45f-48eb-9df0-2f4f4bb46039","Type":"ContainerDied","Data":"fa88c8ccb07a146f0e0252e63ea1a83cba9ac9069f94eaa69216a5405c94798e"} Oct 30 03:25:49.883823 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.883819 4524 scope.go:122] "RemoveContainer" containerID="9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5" Oct 30 03:25:49.885952 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.885929 4524 generic.go:356] "Generic (PLEG): container finished" podID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerID="33fdb4b24adf78945db58aa49b6ee9c855f8e18e60bc716553f3926191e51fe9" exitCode=0 Oct 30 03:25:49.886071 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.885973 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" event={"ID":"f0f3f445-8ef6-4d13-b4e5-73939108c7e9","Type":"ContainerDied","Data":"33fdb4b24adf78945db58aa49b6ee9c855f8e18e60bc716553f3926191e51fe9"} Oct 30 03:25:49.893799 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.893780 4524 scope.go:122] "RemoveContainer" containerID="13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644" Oct 30 03:25:49.902145 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.902124 4524 scope.go:122] "RemoveContainer" containerID="9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5" Oct 30 03:25:49.902405 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:25:49.902387 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5\": container with ID starting with 9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5 not found: ID does not exist" containerID="9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5" Oct 30 03:25:49.902457 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.902417 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5"} err="failed to get container status \"9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5\": rpc error: code = NotFound desc = could not find container \"9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5\": container with ID starting with 9ca8ea81c113cefc14792c90cb66069f8b6ff564f0fe5dc955018f7bc7dcaab5 not found: ID does not exist" Oct 30 03:25:49.902457 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.902434 4524 scope.go:122] "RemoveContainer" containerID="13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644" Oct 30 03:25:49.902709 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:25:49.902692 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644\": container with ID starting with 13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644 not found: ID does not exist" containerID="13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644" Oct 30 03:25:49.902759 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.902715 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644"} err="failed to get container status \"13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644\": rpc error: code = NotFound desc = could not find container \"13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644\": container with ID starting with 13d63c795c0bab62c762893179d246522ebecb8bfc374f77263a890f590fb644 not found: ID does not exist" Oct 30 03:25:49.947154 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.947122 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx"] Oct 30 03:25:49.962179 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.962151 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-5eb02-predictor-7dfbdff689-b7cqx"] Oct 30 03:25:49.977224 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:49.977203 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:25:50.027628 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.027592 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-api-access-zbrxd\" (UniqueName: \"kubernetes.io/projected/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-api-access-zbrxd\") pod \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " Oct 30 03:25:50.027758 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.027657 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-rbac-proxy-sar-config\") pod \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " Oct 30 03:25:50.027758 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.027704 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-proxy-tls\") pod \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\" (UID: \"f0f3f445-8ef6-4d13-b4e5-73939108c7e9\") " Oct 30 03:25:50.031544 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.029883 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-rbac-proxy-sar-config" pod "f0f3f445-8ef6-4d13-b4e5-73939108c7e9" (UID: "f0f3f445-8ef6-4d13-b4e5-73939108c7e9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:25:50.031544 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.030484 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-proxy-tls" pod "f0f3f445-8ef6-4d13-b4e5-73939108c7e9" (UID: "f0f3f445-8ef6-4d13-b4e5-73939108c7e9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:25:50.031544 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.030991 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-api-access-zbrxd" pod "f0f3f445-8ef6-4d13-b4e5-73939108c7e9" (UID: "f0f3f445-8ef6-4d13-b4e5-73939108c7e9"). InnerVolumeSpecName "kube-api-access-zbrxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:25:50.129118 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.129040 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:50.129118 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.129071 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:50.129118 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.129085 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbrxd\" (UniqueName: \"kubernetes.io/projected/f0f3f445-8ef6-4d13-b4e5-73939108c7e9-kube-api-access-zbrxd\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:25:50.891734 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.891704 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" Oct 30 03:25:50.891734 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.891717 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p" event={"ID":"f0f3f445-8ef6-4d13-b4e5-73939108c7e9","Type":"ContainerDied","Data":"953e9c57454b425d690ec87ba9df382912886a8387eb7cf6e13859f8a5c6d6f3"} Oct 30 03:25:50.892276 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.891790 4524 scope.go:122] "RemoveContainer" containerID="abf1a107734651b081eefc70d32edd5b61f9a300a64226021b4edf542316a2b1" Oct 30 03:25:50.899670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.899654 4524 scope.go:122] "RemoveContainer" containerID="33fdb4b24adf78945db58aa49b6ee9c855f8e18e60bc716553f3926191e51fe9" Oct 30 03:25:50.955592 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.955562 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p"] Oct 30 03:25:50.970070 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:50.970040 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-5eb02-predictor-fb7b4f9d8-dzx2p"] Oct 30 03:25:51.019228 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:51.019196 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" path="/var/lib/kubelet/pods/ee96b447-e45f-48eb-9df0-2f4f4bb46039/volumes" Oct 30 03:25:51.019667 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:51.019649 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" path="/var/lib/kubelet/pods/f0f3f445-8ef6-4d13-b4e5-73939108c7e9/volumes" Oct 30 03:25:53.150377 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150342 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6"] Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150646 4524 cpu_manager.go:414] "containerMap: removing container" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kube-rbac-proxy" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150657 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kube-rbac-proxy" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150670 4524 cpu_manager.go:414] "containerMap: removing container" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150676 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150686 4524 cpu_manager.go:414] "containerMap: removing container" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150691 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150697 4524 cpu_manager.go:414] "containerMap: removing container" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kube-rbac-proxy" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150701 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kube-rbac-proxy" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150760 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kube-rbac-proxy" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150768 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kserve-container" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150775 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee96b447-e45f-48eb-9df0-2f4f4bb46039" containerName="kserve-container" Oct 30 03:25:53.150780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.150781 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0f3f445-8ef6-4d13-b4e5-73939108c7e9" containerName="kube-rbac-proxy" Oct 30 03:25:53.155278 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.155262 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.174698 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.174665 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6"] Oct 30 03:25:53.187786 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.187763 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-248e2-predictor-serving-cert\"" Oct 30 03:25:53.187895 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.187801 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"default-dockercfg-dtsbg\"" Oct 30 03:25:53.188748 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.188728 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-248e2-kube-rbac-proxy-sar-config\"" Oct 30 03:25:53.188805 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.188756 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"openshift-service-ca.crt\"" Oct 30 03:25:53.188805 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.188784 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"kube-root-ca.crt\"" Oct 30 03:25:53.249496 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.249470 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4q9k\" (UniqueName: \"kubernetes.io/projected/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-api-access-d4q9k\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.249625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.249503 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c000f96-292b-4d86-9b19-a385f01df9ff-proxy-tls\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.249625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.249606 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.350842 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.350812 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4q9k\" (UniqueName: \"kubernetes.io/projected/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-api-access-d4q9k\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.350842 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.350844 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c000f96-292b-4d86-9b19-a385f01df9ff-proxy-tls\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.351044 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.350875 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.351502 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.351477 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.353261 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.353242 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c000f96-292b-4d86-9b19-a385f01df9ff-proxy-tls\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.378296 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.378276 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4q9k\" (UniqueName: \"kubernetes.io/projected/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-api-access-d4q9k\") pod \"success-200-isvc-248e2-predictor-6b656fb9dc-chld6\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.469354 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.469282 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.542952 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.542924 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9"] Oct 30 03:25:53.547276 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.547256 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.568327 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.568300 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-248e2-kube-rbac-proxy-sar-config\"" Oct 30 03:25:53.568450 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.568430 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-248e2-predictor-serving-cert\"" Oct 30 03:25:53.572101 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.572081 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9"] Oct 30 03:25:53.653968 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.653936 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ff52a39-18f6-4544-ba77-e628edbb33c5-proxy-tls\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.654145 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.653988 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.654213 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.654145 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9d64\" (UniqueName: \"kubernetes.io/projected/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-api-access-k9d64\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.754989 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.754919 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.754989 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.754963 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9d64\" (UniqueName: \"kubernetes.io/projected/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-api-access-k9d64\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.755201 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.755080 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ff52a39-18f6-4544-ba77-e628edbb33c5-proxy-tls\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.755631 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.755607 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.757417 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.757396 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ff52a39-18f6-4544-ba77-e628edbb33c5-proxy-tls\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.762871 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:25:53.762845 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c000f96_292b_4d86_9b19_a385f01df9ff.slice/crio-50b469d7c05f57d9c3f5ad3869f1bf20d22b40f18ea52ff22d34d4252129279c WatchSource:0}: Error finding container 50b469d7c05f57d9c3f5ad3869f1bf20d22b40f18ea52ff22d34d4252129279c: Status 404 returned error can't find the container with id 50b469d7c05f57d9c3f5ad3869f1bf20d22b40f18ea52ff22d34d4252129279c Oct 30 03:25:53.763095 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.763065 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6"] Oct 30 03:25:53.782984 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.782967 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9d64\" (UniqueName: \"kubernetes.io/projected/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-api-access-k9d64\") pod \"error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.862012 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.861982 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:53.906631 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.906166 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" event={"ID":"7c000f96-292b-4d86-9b19-a385f01df9ff","Type":"ContainerStarted","Data":"5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162"} Oct 30 03:25:53.906631 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.906227 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" event={"ID":"7c000f96-292b-4d86-9b19-a385f01df9ff","Type":"ContainerStarted","Data":"64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3"} Oct 30 03:25:53.906631 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.906255 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.906891 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.906646 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" event={"ID":"7c000f96-292b-4d86-9b19-a385f01df9ff","Type":"ContainerStarted","Data":"50b469d7c05f57d9c3f5ad3869f1bf20d22b40f18ea52ff22d34d4252129279c"} Oct 30 03:25:53.906891 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.906676 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:53.907810 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.907778 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused" Oct 30 03:25:53.957486 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:53.957439 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podStartSLOduration=0.957425961 podStartE2EDuration="957.425961ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:25:53.956854154 +0000 UTC m=+1297.078420722" watchObservedRunningTime="2026-10-30 03:25:53.957425961 +0000 UTC m=+1297.078992527" Oct 30 03:25:54.157462 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.157434 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9"] Oct 30 03:25:54.158272 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:25:54.158246 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff52a39_18f6_4544_ba77_e628edbb33c5.slice/crio-6fca7fedb0adcb7512e8510913d299303a52cdf14c3684a10769afcd91c510ca WatchSource:0}: Error finding container 6fca7fedb0adcb7512e8510913d299303a52cdf14c3684a10769afcd91c510ca: Status 404 returned error can't find the container with id 6fca7fedb0adcb7512e8510913d299303a52cdf14c3684a10769afcd91c510ca Oct 30 03:25:54.912998 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.912962 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" event={"ID":"7ff52a39-18f6-4544-ba77-e628edbb33c5","Type":"ContainerStarted","Data":"e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc"} Oct 30 03:25:54.913201 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.913017 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" event={"ID":"7ff52a39-18f6-4544-ba77-e628edbb33c5","Type":"ContainerStarted","Data":"be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c"} Oct 30 03:25:54.913201 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.913046 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" event={"ID":"7ff52a39-18f6-4544-ba77-e628edbb33c5","Type":"ContainerStarted","Data":"6fca7fedb0adcb7512e8510913d299303a52cdf14c3684a10769afcd91c510ca"} Oct 30 03:25:54.913313 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.913225 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:54.913313 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.913263 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:25:54.913313 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.913251 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused" Oct 30 03:25:54.914722 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.914696 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused" Oct 30 03:25:54.963283 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:54.963230 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podStartSLOduration=1.963216426 podStartE2EDuration="1.963216426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:25:54.962266467 +0000 UTC m=+1298.083833033" watchObservedRunningTime="2026-10-30 03:25:54.963216426 +0000 UTC m=+1298.084782993" Oct 30 03:25:55.917605 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:55.917567 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused" Oct 30 03:25:56.736837 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:56.736801 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused" Oct 30 03:25:57.742752 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:57.742714 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused" Oct 30 03:25:59.918289 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:59.918210 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:25:59.918739 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:25:59.918634 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused" Oct 30 03:26:00.922071 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:00.922040 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:26:00.922467 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:00.922443 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused" Oct 30 03:26:06.737661 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:06.737632 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:26:07.743053 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:07.743025 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:26:09.919688 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:09.919654 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused" Oct 30 03:26:10.923253 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:10.923212 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused" Oct 30 03:26:19.339006 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:26:19.338967 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:26:19.919129 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:19.919084 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused" Oct 30 03:26:20.922611 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:20.922574 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused" Oct 30 03:26:29.919068 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:29.919028 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused" Oct 30 03:26:30.922949 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:30.922909 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused" Oct 30 03:26:30.966704 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:30.966676 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2"] Oct 30 03:26:30.966935 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:30.966909 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" containerID="cri-o://93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37" gracePeriod=30 Oct 30 03:26:30.967011 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:30.966963 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kube-rbac-proxy" containerID="cri-o://f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3" gracePeriod=30 Oct 30 03:26:31.091756 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:31.091731 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc"] Oct 30 03:26:31.091971 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:31.091935 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" containerID="cri-o://e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082" gracePeriod=30 Oct 30 03:26:31.092029 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:31.091957 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kube-rbac-proxy" containerID="cri-o://9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04" gracePeriod=30 Oct 30 03:26:31.732350 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:31.732292 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused" Oct 30 03:26:32.064173 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:32.064141 4524 generic.go:356] "Generic (PLEG): container finished" podID="c718652c-c295-4851-816b-6466076e59b6" containerID="9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04" exitCode=0 Oct 30 03:26:32.064549 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:32.064212 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" event={"ID":"c718652c-c295-4851-816b-6466076e59b6","Type":"ContainerDied","Data":"9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04"} Oct 30 03:26:32.066050 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:32.066023 4524 generic.go:356] "Generic (PLEG): container finished" podID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerID="f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3" exitCode=0 Oct 30 03:26:32.066186 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:32.066098 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" event={"ID":"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe","Type":"ContainerDied","Data":"f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3"} Oct 30 03:26:32.737437 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:32.737398 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.34:8643/healthz\": dial tcp 10.133.0.34:8643: connect: connection refused" Oct 30 03:26:34.412835 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.412812 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:26:34.579734 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.579703 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-proxy-tls\") pod \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " Oct 30 03:26:34.579916 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.579765 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-rbac-proxy-sar-config\") pod \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " Oct 30 03:26:34.579916 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.579808 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-api-access-9tkc8\" (UniqueName: \"kubernetes.io/projected/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-api-access-9tkc8\") pod \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\" (UID: \"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe\") " Oct 30 03:26:34.580122 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.580099 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-rbac-proxy-sar-config" pod "1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" (UID: "1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:26:34.581764 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.581739 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-proxy-tls" pod "1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" (UID: "1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:26:34.581856 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.581803 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-api-access-9tkc8" pod "1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" (UID: "1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe"). InnerVolumeSpecName "kube-api-access-9tkc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:26:34.681164 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.681134 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:26:34.681164 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.681164 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:26:34.681329 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.681179 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tkc8\" (UniqueName: \"kubernetes.io/projected/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe-kube-api-access-9tkc8\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:26:34.823496 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.823472 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:26:34.983776 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.983691 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c718652c-c295-4851-816b-6466076e59b6-kube-api-access-kn76g\" (UniqueName: \"kubernetes.io/projected/c718652c-c295-4851-816b-6466076e59b6-kube-api-access-kn76g\") pod \"c718652c-c295-4851-816b-6466076e59b6\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " Oct 30 03:26:34.983776 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.983740 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c718652c-c295-4851-816b-6466076e59b6-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c718652c-c295-4851-816b-6466076e59b6-proxy-tls\") pod \"c718652c-c295-4851-816b-6466076e59b6\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " Oct 30 03:26:34.983776 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.983772 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c718652c-c295-4851-816b-6466076e59b6-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c718652c-c295-4851-816b-6466076e59b6-kube-rbac-proxy-sar-config\") pod \"c718652c-c295-4851-816b-6466076e59b6\" (UID: \"c718652c-c295-4851-816b-6466076e59b6\") " Oct 30 03:26:34.984191 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.984169 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c718652c-c295-4851-816b-6466076e59b6-kube-rbac-proxy-sar-config" pod "c718652c-c295-4851-816b-6466076e59b6" (UID: "c718652c-c295-4851-816b-6466076e59b6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:26:34.985879 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.985859 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c718652c-c295-4851-816b-6466076e59b6-kube-api-access-kn76g" pod "c718652c-c295-4851-816b-6466076e59b6" (UID: "c718652c-c295-4851-816b-6466076e59b6"). InnerVolumeSpecName "kube-api-access-kn76g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:26:34.985939 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:34.985879 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c718652c-c295-4851-816b-6466076e59b6-proxy-tls" pod "c718652c-c295-4851-816b-6466076e59b6" (UID: "c718652c-c295-4851-816b-6466076e59b6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:26:35.078132 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.078100 4524 generic.go:356] "Generic (PLEG): container finished" podID="c718652c-c295-4851-816b-6466076e59b6" containerID="e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082" exitCode=0 Oct 30 03:26:35.078292 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.078153 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" Oct 30 03:26:35.078292 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.078177 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" event={"ID":"c718652c-c295-4851-816b-6466076e59b6","Type":"ContainerDied","Data":"e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082"} Oct 30 03:26:35.078292 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.078243 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc" event={"ID":"c718652c-c295-4851-816b-6466076e59b6","Type":"ContainerDied","Data":"076f75e7f60be1ec24fdeca4afe76a2f55225b9df70b94b12e92ae67456f9c33"} Oct 30 03:26:35.078292 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.078274 4524 scope.go:122] "RemoveContainer" containerID="9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04" Oct 30 03:26:35.079759 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.079735 4524 generic.go:356] "Generic (PLEG): container finished" podID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerID="93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37" exitCode=0 Oct 30 03:26:35.079850 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.079804 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" Oct 30 03:26:35.079850 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.079815 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" event={"ID":"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe","Type":"ContainerDied","Data":"93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37"} Oct 30 03:26:35.079931 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.079858 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2" event={"ID":"1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe","Type":"ContainerDied","Data":"cf32fb01983ee6581e01b83853d65aac2629cc5e317630fb8ed0d29709290868"} Oct 30 03:26:35.084824 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.084798 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn76g\" (UniqueName: \"kubernetes.io/projected/c718652c-c295-4851-816b-6466076e59b6-kube-api-access-kn76g\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:26:35.084824 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.084823 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c718652c-c295-4851-816b-6466076e59b6-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:26:35.084956 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.084837 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c718652c-c295-4851-816b-6466076e59b6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:26:35.086431 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.086416 4524 scope.go:122] "RemoveContainer" containerID="e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082" Oct 30 03:26:35.093245 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.093210 4524 scope.go:122] "RemoveContainer" containerID="9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04" Oct 30 03:26:35.093536 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:26:35.093501 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04\": container with ID starting with 9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04 not found: ID does not exist" containerID="9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04" Oct 30 03:26:35.093616 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.093539 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04"} err="failed to get container status \"9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04\": rpc error: code = NotFound desc = could not find container \"9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04\": container with ID starting with 9f2c54c2f63009a3958abcacca8985b7ecc29a5ec7480df777f63b43652c4a04 not found: ID does not exist" Oct 30 03:26:35.093616 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.093568 4524 scope.go:122] "RemoveContainer" containerID="e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082" Oct 30 03:26:35.093808 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:26:35.093790 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082\": container with ID starting with e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082 not found: ID does not exist" containerID="e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082" Oct 30 03:26:35.093842 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.093815 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082"} err="failed to get container status \"e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082\": rpc error: code = NotFound desc = could not find container \"e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082\": container with ID starting with e20900ae80338757f2df58b9f4bb01165e558bdf3a2963228c82df508e3c3082 not found: ID does not exist" Oct 30 03:26:35.093842 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.093830 4524 scope.go:122] "RemoveContainer" containerID="f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3" Oct 30 03:26:35.100336 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.100319 4524 scope.go:122] "RemoveContainer" containerID="93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37" Oct 30 03:26:35.106688 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.106670 4524 scope.go:122] "RemoveContainer" containerID="f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3" Oct 30 03:26:35.106940 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:26:35.106923 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3\": container with ID starting with f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3 not found: ID does not exist" containerID="f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3" Oct 30 03:26:35.106987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.106946 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3"} err="failed to get container status \"f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3\": rpc error: code = NotFound desc = could not find container \"f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3\": container with ID starting with f6673c840d5cef9f2c88acbf711f2f0c2a5bbb377d94e292f8180efee5ff4ab3 not found: ID does not exist" Oct 30 03:26:35.106987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.106965 4524 scope.go:122] "RemoveContainer" containerID="93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37" Oct 30 03:26:35.107190 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:26:35.107173 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37\": container with ID starting with 93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37 not found: ID does not exist" containerID="93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37" Oct 30 03:26:35.107244 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.107198 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37"} err="failed to get container status \"93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37\": rpc error: code = NotFound desc = could not find container \"93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37\": container with ID starting with 93fd5289c087fbb912d8f01f5a99b1fa3b18c5f8091485594f07bcdf4cf9fd37 not found: ID does not exist" Oct 30 03:26:35.138602 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.138569 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2"] Oct 30 03:26:35.141705 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.141684 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-a3800-predictor-7c6f7bf6c9-g7qh2"] Oct 30 03:26:35.183484 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.183457 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc"] Oct 30 03:26:35.191538 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:35.191501 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-a3800-predictor-cc47c895d-6r8mc"] Oct 30 03:26:36.512339 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512306 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5"] Oct 30 03:26:36.512740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512647 4524 cpu_manager.go:414] "containerMap: removing container" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" Oct 30 03:26:36.512740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512659 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" Oct 30 03:26:36.512740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512671 4524 cpu_manager.go:414] "containerMap: removing container" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kube-rbac-proxy" Oct 30 03:26:36.512740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512677 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kube-rbac-proxy" Oct 30 03:26:36.512740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512696 4524 cpu_manager.go:414] "containerMap: removing container" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kube-rbac-proxy" Oct 30 03:26:36.512740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512702 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kube-rbac-proxy" Oct 30 03:26:36.512740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512709 4524 cpu_manager.go:414] "containerMap: removing container" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" Oct 30 03:26:36.512740 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512728 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" Oct 30 03:26:36.512971 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512772 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kube-rbac-proxy" Oct 30 03:26:36.512971 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512780 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kserve-container" Oct 30 03:26:36.512971 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512787 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" containerName="kserve-container" Oct 30 03:26:36.512971 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.512792 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c718652c-c295-4851-816b-6466076e59b6" containerName="kube-rbac-proxy" Oct 30 03:26:36.517031 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.517014 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.539915 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.539890 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5"] Oct 30 03:26:36.549262 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.549238 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"success-200-isvc-81fd6-predictor-serving-cert\"" Oct 30 03:26:36.549386 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.549286 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"default-dockercfg-tlgnp\"" Oct 30 03:26:36.549386 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.549282 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"success-200-isvc-81fd6-kube-rbac-proxy-sar-config\"" Oct 30 03:26:36.549386 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.549304 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"openshift-service-ca.crt\"" Oct 30 03:26:36.549534 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.549510 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"kube-root-ca.crt\"" Oct 30 03:26:36.697250 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.697219 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c817a3e1-558b-49f4-a49d-58989edc73bf-proxy-tls\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.697390 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.697257 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqhjt\" (UniqueName: \"kubernetes.io/projected/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-api-access-sqhjt\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.697390 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.697344 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.798150 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.798075 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c817a3e1-558b-49f4-a49d-58989edc73bf-proxy-tls\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.798150 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.798116 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqhjt\" (UniqueName: \"kubernetes.io/projected/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-api-access-sqhjt\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.798324 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.798166 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.798864 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.798839 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.800554 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.800528 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c817a3e1-558b-49f4-a49d-58989edc73bf-proxy-tls\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.825595 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.825574 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqhjt\" (UniqueName: \"kubernetes.io/projected/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-api-access-sqhjt\") pod \"success-200-isvc-81fd6-predictor-79ccc896b4-v47d5\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.831265 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.831243 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:36.912797 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.912772 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249"] Oct 30 03:26:36.916293 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.916277 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:36.938397 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.938374 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"error-404-isvc-81fd6-predictor-serving-cert\"" Oct 30 03:26:36.938550 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.938435 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"error-404-isvc-81fd6-kube-rbac-proxy-sar-config\"" Oct 30 03:26:36.939114 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:36.939090 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249"] Oct 30 03:26:37.019451 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.019421 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe" path="/var/lib/kubelet/pods/1c00590c-fd7b-45ca-92d4-fcbdff7fc8fe/volumes" Oct 30 03:26:37.019933 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.019914 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c718652c-c295-4851-816b-6466076e59b6" path="/var/lib/kubelet/pods/c718652c-c295-4851-816b-6466076e59b6/volumes" Oct 30 03:26:37.100559 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.100535 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e25b0a42-a5f5-4828-963b-693385ea1043-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.100693 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.100605 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47w6w\" (UniqueName: \"kubernetes.io/projected/e25b0a42-a5f5-4828-963b-693385ea1043-kube-api-access-47w6w\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.100693 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.100660 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e25b0a42-a5f5-4828-963b-693385ea1043-proxy-tls\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.126751 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.126721 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5"] Oct 30 03:26:37.127373 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:26:37.127332 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc817a3e1_558b_49f4_a49d_58989edc73bf.slice/crio-bf550079b56142cfcc441765ebcba21a9f5684aa42532b67b32c898dd22f78dc WatchSource:0}: Error finding container bf550079b56142cfcc441765ebcba21a9f5684aa42532b67b32c898dd22f78dc: Status 404 returned error can't find the container with id bf550079b56142cfcc441765ebcba21a9f5684aa42532b67b32c898dd22f78dc Oct 30 03:26:37.201370 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.201347 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e25b0a42-a5f5-4828-963b-693385ea1043-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.201462 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.201381 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47w6w\" (UniqueName: \"kubernetes.io/projected/e25b0a42-a5f5-4828-963b-693385ea1043-kube-api-access-47w6w\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.201462 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.201407 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e25b0a42-a5f5-4828-963b-693385ea1043-proxy-tls\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.202189 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.202164 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e25b0a42-a5f5-4828-963b-693385ea1043-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.203473 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.203446 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e25b0a42-a5f5-4828-963b-693385ea1043-proxy-tls\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.228812 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.228786 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47w6w\" (UniqueName: \"kubernetes.io/projected/e25b0a42-a5f5-4828-963b-693385ea1043-kube-api-access-47w6w\") pod \"error-404-isvc-81fd6-predictor-5c74697485-zb249\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.230418 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.230395 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:37.531592 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:26:37.531559 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25b0a42_a5f5_4828_963b_693385ea1043.slice/crio-270578552abb7723d876458b6570d9dd04099af9c896dcc0d9fc06369a6488fe WatchSource:0}: Error finding container 270578552abb7723d876458b6570d9dd04099af9c896dcc0d9fc06369a6488fe: Status 404 returned error can't find the container with id 270578552abb7723d876458b6570d9dd04099af9c896dcc0d9fc06369a6488fe Oct 30 03:26:37.531995 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:37.531680 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249"] Oct 30 03:26:38.094955 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.094916 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" event={"ID":"c817a3e1-558b-49f4-a49d-58989edc73bf","Type":"ContainerStarted","Data":"50fd136ba4a60be0454fe61620d8dcbeb4390a17971449695b5733b664d9e402"} Oct 30 03:26:38.095119 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.094988 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" event={"ID":"c817a3e1-558b-49f4-a49d-58989edc73bf","Type":"ContainerStarted","Data":"756693f824028e02d8107688481155a75e44528cf556be4c06edc9da76a1dffc"} Oct 30 03:26:38.095119 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.095021 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" event={"ID":"c817a3e1-558b-49f4-a49d-58989edc73bf","Type":"ContainerStarted","Data":"bf550079b56142cfcc441765ebcba21a9f5684aa42532b67b32c898dd22f78dc"} Oct 30 03:26:38.095119 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.095053 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:38.095119 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.095090 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:38.096338 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.096313 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused" Oct 30 03:26:38.096862 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.096833 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" event={"ID":"e25b0a42-a5f5-4828-963b-693385ea1043","Type":"ContainerStarted","Data":"01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea"} Oct 30 03:26:38.096957 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.096885 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" event={"ID":"e25b0a42-a5f5-4828-963b-693385ea1043","Type":"ContainerStarted","Data":"963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66"} Oct 30 03:26:38.096957 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.096908 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" event={"ID":"e25b0a42-a5f5-4828-963b-693385ea1043","Type":"ContainerStarted","Data":"270578552abb7723d876458b6570d9dd04099af9c896dcc0d9fc06369a6488fe"} Oct 30 03:26:38.096957 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.096931 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:38.144927 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.144884 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podStartSLOduration=2.144872735 podStartE2EDuration="2.144872735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:26:38.144287331 +0000 UTC m=+1341.265853898" watchObservedRunningTime="2026-10-30 03:26:38.144872735 +0000 UTC m=+1341.266439300" Oct 30 03:26:38.194505 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:38.194462 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podStartSLOduration=2.19445037 podStartE2EDuration="2.19445037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:26:38.193622647 +0000 UTC m=+1341.315189233" watchObservedRunningTime="2026-10-30 03:26:38.19445037 +0000 UTC m=+1341.316016937" Oct 30 03:26:39.101070 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:39.101023 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:39.101473 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:39.101215 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused" Oct 30 03:26:39.102472 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:39.102451 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused" Oct 30 03:26:39.918841 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:39.918802 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused" Oct 30 03:26:40.105458 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:40.105412 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused" Oct 30 03:26:40.922493 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:40.922457 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused" Oct 30 03:26:44.105635 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:44.105602 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:26:44.106223 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:44.106195 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused" Oct 30 03:26:45.110095 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:45.110067 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:26:45.110593 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:45.110480 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused" Oct 30 03:26:49.339421 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:26:49.339384 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:26:49.920151 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:49.920124 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:26:50.923045 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:50.923017 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:26:54.106346 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:54.106312 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused" Oct 30 03:26:55.110551 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:26:55.110494 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused" Oct 30 03:27:04.106162 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:27:04.106122 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused" Oct 30 03:27:05.111091 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:27:05.111053 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused" Oct 30 03:27:14.106249 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:27:14.106209 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused" Oct 30 03:27:15.110638 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:27:15.110595 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused" Oct 30 03:27:19.340114 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:27:19.340078 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:27:24.106768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:27:24.106720 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused" Oct 30 03:27:25.110752 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:27:25.110712 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused" Oct 30 03:27:34.106672 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:27:34.106642 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:27:35.111691 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:27:35.111665 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:27:49.337078 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:27:49.337038 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:28:19.336056 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:28:19.336018 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:28:49.336282 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:28:49.336250 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:29:19.336835 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:29:19.336801 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:29:49.336849 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:29:49.336810 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:30:19.336671 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:30:19.336637 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:30:49.336744 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:30:49.336709 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:31:19.340345 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:31:19.340307 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:31:49.336183 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:31:49.336149 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:32:19.337176 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:32:19.337104 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:32:49.337042 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:32:49.337006 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:33:19.335902 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:33:19.335870 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:33:49.336829 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:33:49.336792 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:34:19.337331 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:34:19.337235 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:34:49.336282 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:34:49.336241 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:35:18.898811 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:18.898733 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9"] Oct 30 03:35:18.899301 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:18.898944 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" containerID="cri-o://be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c" gracePeriod=30 Oct 30 03:35:18.899301 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:18.898969 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kube-rbac-proxy" containerID="cri-o://e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc" gracePeriod=30 Oct 30 03:35:19.042959 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:19.042931 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6"] Oct 30 03:35:19.043148 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:19.043128 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" containerID="cri-o://64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3" gracePeriod=30 Oct 30 03:35:19.043206 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:19.043166 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kube-rbac-proxy" containerID="cri-o://5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162" gracePeriod=30 Oct 30 03:35:19.109714 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:19.109678 4524 generic.go:356] "Generic (PLEG): container finished" podID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerID="e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc" exitCode=0 Oct 30 03:35:19.109876 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:19.109734 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" event={"ID":"7ff52a39-18f6-4544-ba77-e628edbb33c5","Type":"ContainerDied","Data":"e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc"} Oct 30 03:35:19.338527 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:35:19.338479 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:35:19.913793 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:19.913742 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.35:8643/healthz\": dial tcp 10.133.0.35:8643: connect: connection refused" Oct 30 03:35:19.919162 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:19.919137 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused" Oct 30 03:35:20.115406 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:20.115368 4524 generic.go:356] "Generic (PLEG): container finished" podID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerID="5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162" exitCode=0 Oct 30 03:35:20.115586 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:20.115438 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" event={"ID":"7c000f96-292b-4d86-9b19-a385f01df9ff","Type":"ContainerDied","Data":"5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162"} Oct 30 03:35:20.918553 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:20.918498 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused" Oct 30 03:35:20.922903 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:20.922883 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused" Oct 30 03:35:22.051657 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.051631 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:35:22.089794 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.089748 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:35:22.123266 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.123238 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c000f96-292b-4d86-9b19-a385f01df9ff-proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c000f96-292b-4d86-9b19-a385f01df9ff-proxy-tls\") pod \"7c000f96-292b-4d86-9b19-a385f01df9ff\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " Oct 30 03:35:22.123404 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.123295 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-rbac-proxy-sar-config\") pod \"7c000f96-292b-4d86-9b19-a385f01df9ff\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " Oct 30 03:35:22.123404 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.123327 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ff52a39-18f6-4544-ba77-e628edbb33c5-proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ff52a39-18f6-4544-ba77-e628edbb33c5-proxy-tls\") pod \"7ff52a39-18f6-4544-ba77-e628edbb33c5\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " Oct 30 03:35:22.123404 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.123362 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-rbac-proxy-sar-config\") pod \"7ff52a39-18f6-4544-ba77-e628edbb33c5\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " Oct 30 03:35:22.123404 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.123389 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-api-access-d4q9k\" (UniqueName: \"kubernetes.io/projected/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-api-access-d4q9k\") pod \"7c000f96-292b-4d86-9b19-a385f01df9ff\" (UID: \"7c000f96-292b-4d86-9b19-a385f01df9ff\") " Oct 30 03:35:22.123637 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.123438 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-api-access-k9d64\" (UniqueName: \"kubernetes.io/projected/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-api-access-k9d64\") pod \"7ff52a39-18f6-4544-ba77-e628edbb33c5\" (UID: \"7ff52a39-18f6-4544-ba77-e628edbb33c5\") " Oct 30 03:35:22.123763 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.123731 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-rbac-proxy-sar-config" pod "7ff52a39-18f6-4544-ba77-e628edbb33c5" (UID: "7ff52a39-18f6-4544-ba77-e628edbb33c5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:35:22.123763 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.123735 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-rbac-proxy-sar-config" pod "7c000f96-292b-4d86-9b19-a385f01df9ff" (UID: "7c000f96-292b-4d86-9b19-a385f01df9ff"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:35:22.125209 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125180 4524 generic.go:356] "Generic (PLEG): container finished" podID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerID="64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3" exitCode=0 Oct 30 03:35:22.125318 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125300 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" Oct 30 03:35:22.125388 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125326 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" event={"ID":"7c000f96-292b-4d86-9b19-a385f01df9ff","Type":"ContainerDied","Data":"64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3"} Oct 30 03:35:22.125454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125394 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6" event={"ID":"7c000f96-292b-4d86-9b19-a385f01df9ff","Type":"ContainerDied","Data":"50b469d7c05f57d9c3f5ad3869f1bf20d22b40f18ea52ff22d34d4252129279c"} Oct 30 03:35:22.125454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125433 4524 scope.go:122] "RemoveContainer" containerID="5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162" Oct 30 03:35:22.125912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125888 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-api-access-k9d64" pod "7ff52a39-18f6-4544-ba77-e628edbb33c5" (UID: "7ff52a39-18f6-4544-ba77-e628edbb33c5"). InnerVolumeSpecName "kube-api-access-k9d64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:35:22.125912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125906 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-api-access-d4q9k" pod "7c000f96-292b-4d86-9b19-a385f01df9ff" (UID: "7c000f96-292b-4d86-9b19-a385f01df9ff"). InnerVolumeSpecName "kube-api-access-d4q9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:35:22.126056 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125913 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c000f96-292b-4d86-9b19-a385f01df9ff-proxy-tls" pod "7c000f96-292b-4d86-9b19-a385f01df9ff" (UID: "7c000f96-292b-4d86-9b19-a385f01df9ff"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:35:22.126056 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.125892 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ff52a39-18f6-4544-ba77-e628edbb33c5-proxy-tls" pod "7ff52a39-18f6-4544-ba77-e628edbb33c5" (UID: "7ff52a39-18f6-4544-ba77-e628edbb33c5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:35:22.127173 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.127149 4524 generic.go:356] "Generic (PLEG): container finished" podID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerID="be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c" exitCode=0 Oct 30 03:35:22.127267 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.127201 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" event={"ID":"7ff52a39-18f6-4544-ba77-e628edbb33c5","Type":"ContainerDied","Data":"be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c"} Oct 30 03:35:22.127267 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.127206 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" Oct 30 03:35:22.127267 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.127248 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9" event={"ID":"7ff52a39-18f6-4544-ba77-e628edbb33c5","Type":"ContainerDied","Data":"6fca7fedb0adcb7512e8510913d299303a52cdf14c3684a10769afcd91c510ca"} Oct 30 03:35:22.133293 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.133274 4524 scope.go:122] "RemoveContainer" containerID="64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3" Oct 30 03:35:22.140554 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.140538 4524 scope.go:122] "RemoveContainer" containerID="5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162" Oct 30 03:35:22.140818 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:35:22.140801 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162\": container with ID starting with 5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162 not found: ID does not exist" containerID="5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162" Oct 30 03:35:22.140885 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.140829 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162"} err="failed to get container status \"5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162\": rpc error: code = NotFound desc = could not find container \"5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162\": container with ID starting with 5ca4145fb8b2eb698d164f592433d87559415a3ae2f7082ce1fae95ceebe8162 not found: ID does not exist" Oct 30 03:35:22.140885 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.140854 4524 scope.go:122] "RemoveContainer" containerID="64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3" Oct 30 03:35:22.141117 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:35:22.141100 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3\": container with ID starting with 64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3 not found: ID does not exist" containerID="64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3" Oct 30 03:35:22.141159 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.141123 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3"} err="failed to get container status \"64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3\": rpc error: code = NotFound desc = could not find container \"64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3\": container with ID starting with 64dfa6204dfcc21c64a4374398ef9cc9d6d19f11e1320c222aa6d0f0f129e3b3 not found: ID does not exist" Oct 30 03:35:22.141159 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.141138 4524 scope.go:122] "RemoveContainer" containerID="e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc" Oct 30 03:35:22.147599 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.147584 4524 scope.go:122] "RemoveContainer" containerID="be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c" Oct 30 03:35:22.155070 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.155049 4524 scope.go:122] "RemoveContainer" containerID="e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc" Oct 30 03:35:22.155354 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:35:22.155337 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc\": container with ID starting with e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc not found: ID does not exist" containerID="e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc" Oct 30 03:35:22.155436 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.155361 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc"} err="failed to get container status \"e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc\": rpc error: code = NotFound desc = could not find container \"e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc\": container with ID starting with e39737685038d54cc7dbb077e7e9c1ff33279ba1c3dddc67da3183dd69035ccc not found: ID does not exist" Oct 30 03:35:22.155436 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.155375 4524 scope.go:122] "RemoveContainer" containerID="be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c" Oct 30 03:35:22.155680 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:35:22.155661 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c\": container with ID starting with be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c not found: ID does not exist" containerID="be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c" Oct 30 03:35:22.155726 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.155688 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c"} err="failed to get container status \"be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c\": rpc error: code = NotFound desc = could not find container \"be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c\": container with ID starting with be1fe1d4dd43fe376669b534d28d27e349a9bb9299d9624790ab6a1ee7d9f77c not found: ID does not exist" Oct 30 03:35:22.196586 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.196562 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9"] Oct 30 03:35:22.205032 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.205008 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-248e2-predictor-5c48dbdf78-rrsc9"] Oct 30 03:35:22.224684 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.224664 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:35:22.224684 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.224683 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ff52a39-18f6-4544-ba77-e628edbb33c5-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:35:22.224786 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.224693 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:35:22.224786 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.224704 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4q9k\" (UniqueName: \"kubernetes.io/projected/7c000f96-292b-4d86-9b19-a385f01df9ff-kube-api-access-d4q9k\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:35:22.224786 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.224713 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9d64\" (UniqueName: \"kubernetes.io/projected/7ff52a39-18f6-4544-ba77-e628edbb33c5-kube-api-access-k9d64\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:35:22.224786 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.224721 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c000f96-292b-4d86-9b19-a385f01df9ff-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:35:22.477647 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.477619 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6"] Oct 30 03:35:22.485355 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:22.485329 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-248e2-predictor-6b656fb9dc-chld6"] Oct 30 03:35:23.019408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:23.019378 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" path="/var/lib/kubelet/pods/7c000f96-292b-4d86-9b19-a385f01df9ff/volumes" Oct 30 03:35:23.019826 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:23.019812 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" path="/var/lib/kubelet/pods/7ff52a39-18f6-4544-ba77-e628edbb33c5/volumes" Oct 30 03:35:24.444955 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.444924 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888"] Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445217 4524 cpu_manager.go:414] "containerMap: removing container" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kube-rbac-proxy" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445228 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kube-rbac-proxy" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445257 4524 cpu_manager.go:414] "containerMap: removing container" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445263 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445270 4524 cpu_manager.go:414] "containerMap: removing container" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445275 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445283 4524 cpu_manager.go:414] "containerMap: removing container" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kube-rbac-proxy" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445287 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kube-rbac-proxy" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445333 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kube-rbac-proxy" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445344 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ff52a39-18f6-4544-ba77-e628edbb33c5" containerName="kserve-container" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445351 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kserve-container" Oct 30 03:35:24.445408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.445356 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c000f96-292b-4d86-9b19-a385f01df9ff" containerName="kube-rbac-proxy" Oct 30 03:35:24.449616 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.449597 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.484418 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.484394 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-32dc6-kube-rbac-proxy-sar-config\"" Oct 30 03:35:24.484418 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.484408 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"kube-root-ca.crt\"" Oct 30 03:35:24.484667 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.484437 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"openshift-service-ca.crt\"" Oct 30 03:35:24.490481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.484947 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-32dc6-predictor-serving-cert\"" Oct 30 03:35:24.490481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.485025 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"default-dockercfg-dtsbg\"" Oct 30 03:35:24.490481 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.487228 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888"] Oct 30 03:35:24.541807 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.541782 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd2a247-b042-4530-9b89-874c1ac12780-proxy-tls\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.541943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.541816 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd2a247-b042-4530-9b89-874c1ac12780-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.541943 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.541840 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6nj8\" (UniqueName: \"kubernetes.io/projected/4fd2a247-b042-4530-9b89-874c1ac12780-kube-api-access-p6nj8\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.643119 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.643089 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd2a247-b042-4530-9b89-874c1ac12780-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.643259 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.643130 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6nj8\" (UniqueName: \"kubernetes.io/projected/4fd2a247-b042-4530-9b89-874c1ac12780-kube-api-access-p6nj8\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.643259 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.643180 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd2a247-b042-4530-9b89-874c1ac12780-proxy-tls\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.643812 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.643788 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd2a247-b042-4530-9b89-874c1ac12780-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.645587 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.645559 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd2a247-b042-4530-9b89-874c1ac12780-proxy-tls\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.671185 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.671162 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6nj8\" (UniqueName: \"kubernetes.io/projected/4fd2a247-b042-4530-9b89-874c1ac12780-kube-api-access-p6nj8\") pod \"success-200-isvc-32dc6-predictor-565d78d569-jz888\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.764157 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.764095 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:24.781577 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.781554 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9"] Oct 30 03:35:24.785381 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.785358 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.806391 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.806359 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-32dc6-kube-rbac-proxy-sar-config\"" Oct 30 03:35:24.806691 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.806668 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-32dc6-predictor-serving-cert\"" Oct 30 03:35:24.807320 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.807299 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9"] Oct 30 03:35:24.845055 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.845019 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m7ml\" (UniqueName: \"kubernetes.io/projected/113785af-1748-4ea5-982a-b872963ffc4e-kube-api-access-7m7ml\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.845214 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.845140 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/113785af-1748-4ea5-982a-b872963ffc4e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.845214 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.845205 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113785af-1748-4ea5-982a-b872963ffc4e-proxy-tls\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.946535 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.946490 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/113785af-1748-4ea5-982a-b872963ffc4e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.946535 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.946542 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113785af-1748-4ea5-982a-b872963ffc4e-proxy-tls\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.946789 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.946582 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m7ml\" (UniqueName: \"kubernetes.io/projected/113785af-1748-4ea5-982a-b872963ffc4e-kube-api-access-7m7ml\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.947123 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.947101 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/113785af-1748-4ea5-982a-b872963ffc4e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.948906 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.948885 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113785af-1748-4ea5-982a-b872963ffc4e-proxy-tls\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:24.974188 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:24.974161 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m7ml\" (UniqueName: \"kubernetes.io/projected/113785af-1748-4ea5-982a-b872963ffc4e-kube-api-access-7m7ml\") pod \"error-404-isvc-32dc6-predictor-6594bd75b4-m24n9\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:25.062322 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:35:25.062296 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd2a247_b042_4530_9b89_874c1ac12780.slice/crio-b629f1b3746109224d44dd9c997bc2b7efe3b99dde9104338728184a1cd0d0e2 WatchSource:0}: Error finding container b629f1b3746109224d44dd9c997bc2b7efe3b99dde9104338728184a1cd0d0e2: Status 404 returned error can't find the container with id b629f1b3746109224d44dd9c997bc2b7efe3b99dde9104338728184a1cd0d0e2 Oct 30 03:35:25.062794 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:25.062771 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888"] Oct 30 03:35:25.099948 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:25.099868 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:25.142531 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:25.142491 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" event={"ID":"4fd2a247-b042-4530-9b89-874c1ac12780","Type":"ContainerStarted","Data":"c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91"} Oct 30 03:35:25.142656 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:25.142591 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" event={"ID":"4fd2a247-b042-4530-9b89-874c1ac12780","Type":"ContainerStarted","Data":"b629f1b3746109224d44dd9c997bc2b7efe3b99dde9104338728184a1cd0d0e2"} Oct 30 03:35:25.401900 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:35:25.401861 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113785af_1748_4ea5_982a_b872963ffc4e.slice/crio-62e19e73cdaa950d4b09f307a5411a506924f8bb439cfae94577aa42737a6e23 WatchSource:0}: Error finding container 62e19e73cdaa950d4b09f307a5411a506924f8bb439cfae94577aa42737a6e23: Status 404 returned error can't find the container with id 62e19e73cdaa950d4b09f307a5411a506924f8bb439cfae94577aa42737a6e23 Oct 30 03:35:25.402279 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:25.402258 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9"] Oct 30 03:35:26.148488 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.148446 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" event={"ID":"113785af-1748-4ea5-982a-b872963ffc4e","Type":"ContainerStarted","Data":"d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7"} Oct 30 03:35:26.148980 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.148511 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" event={"ID":"113785af-1748-4ea5-982a-b872963ffc4e","Type":"ContainerStarted","Data":"33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3"} Oct 30 03:35:26.148980 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.148557 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" event={"ID":"113785af-1748-4ea5-982a-b872963ffc4e","Type":"ContainerStarted","Data":"62e19e73cdaa950d4b09f307a5411a506924f8bb439cfae94577aa42737a6e23"} Oct 30 03:35:26.148980 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.148581 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:26.148980 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.148611 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:26.150342 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.150316 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused" Oct 30 03:35:26.150551 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.150501 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" event={"ID":"4fd2a247-b042-4530-9b89-874c1ac12780","Type":"ContainerStarted","Data":"b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3"} Oct 30 03:35:26.150682 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.150579 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:26.150682 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.150602 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:26.151632 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.151609 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused" Oct 30 03:35:26.198025 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.197979 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podStartSLOduration=2.197963763 podStartE2EDuration="2.197963763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:35:26.197596887 +0000 UTC m=+1869.319163451" watchObservedRunningTime="2026-10-30 03:35:26.197963763 +0000 UTC m=+1869.319530331" Oct 30 03:35:26.247365 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:26.247316 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podStartSLOduration=2.247303489 podStartE2EDuration="2.247303489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:35:26.246986991 +0000 UTC m=+1869.368553567" watchObservedRunningTime="2026-10-30 03:35:26.247303489 +0000 UTC m=+1869.368870056" Oct 30 03:35:27.154335 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:27.154295 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused" Oct 30 03:35:27.154745 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:27.154367 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused" Oct 30 03:35:32.159252 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:32.159223 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:35:32.159643 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:32.159348 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:35:32.159738 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:32.159711 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused" Oct 30 03:35:32.159790 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:32.159771 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused" Oct 30 03:35:42.159757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:42.159716 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused" Oct 30 03:35:42.160157 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:42.159716 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused" Oct 30 03:35:49.338345 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:35:49.338306 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:35:52.160299 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:52.160262 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused" Oct 30 03:35:52.160701 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:35:52.160261 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused" Oct 30 03:36:02.018869 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.018835 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249"] Oct 30 03:36:02.019393 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.019039 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" containerID="cri-o://963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66" gracePeriod=30 Oct 30 03:36:02.019393 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.019169 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kube-rbac-proxy" containerID="cri-o://01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea" gracePeriod=30 Oct 30 03:36:02.028731 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:02.028703 4524 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25b0a42_a5f5_4828_963b_693385ea1043.slice/crio-01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea.scope\": RecentStats: unable to find data in memory cache]" Oct 30 03:36:02.150910 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.150880 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5"] Oct 30 03:36:02.151139 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.151102 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" containerID="cri-o://756693f824028e02d8107688481155a75e44528cf556be4c06edc9da76a1dffc" gracePeriod=30 Oct 30 03:36:02.151208 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.151134 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kube-rbac-proxy" containerID="cri-o://50fd136ba4a60be0454fe61620d8dcbeb4390a17971449695b5733b664d9e402" gracePeriod=30 Oct 30 03:36:02.159948 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.159911 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused" Oct 30 03:36:02.160058 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.159911 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused" Oct 30 03:36:02.305997 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.305964 4524 generic.go:356] "Generic (PLEG): container finished" podID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerID="50fd136ba4a60be0454fe61620d8dcbeb4390a17971449695b5733b664d9e402" exitCode=0 Oct 30 03:36:02.306143 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.306038 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" event={"ID":"c817a3e1-558b-49f4-a49d-58989edc73bf","Type":"ContainerDied","Data":"50fd136ba4a60be0454fe61620d8dcbeb4390a17971449695b5733b664d9e402"} Oct 30 03:36:02.307816 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.307793 4524 generic.go:356] "Generic (PLEG): container finished" podID="e25b0a42-a5f5-4828-963b-693385ea1043" containerID="01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea" exitCode=0 Oct 30 03:36:02.307922 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:02.307835 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" event={"ID":"e25b0a42-a5f5-4828-963b-693385ea1043","Type":"ContainerDied","Data":"01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea"} Oct 30 03:36:04.101577 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:04.101533 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.37:8643/healthz\": dial tcp 10.133.0.37:8643: connect: connection refused" Oct 30 03:36:04.106930 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:04.106907 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused" Oct 30 03:36:05.106617 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.106580 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.38:8643/healthz\": dial tcp 10.133.0.38:8643: connect: connection refused" Oct 30 03:36:05.110959 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.110936 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused" Oct 30 03:36:05.321569 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.321542 4524 generic.go:356] "Generic (PLEG): container finished" podID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerID="756693f824028e02d8107688481155a75e44528cf556be4c06edc9da76a1dffc" exitCode=0 Oct 30 03:36:05.321691 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.321597 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" event={"ID":"c817a3e1-558b-49f4-a49d-58989edc73bf","Type":"ContainerDied","Data":"756693f824028e02d8107688481155a75e44528cf556be4c06edc9da76a1dffc"} Oct 30 03:36:05.511498 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.511475 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:36:05.586829 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.586791 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c817a3e1-558b-49f4-a49d-58989edc73bf-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c817a3e1-558b-49f4-a49d-58989edc73bf-proxy-tls\") pod \"c817a3e1-558b-49f4-a49d-58989edc73bf\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " Oct 30 03:36:05.586991 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.586847 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-rbac-proxy-sar-config\") pod \"c817a3e1-558b-49f4-a49d-58989edc73bf\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " Oct 30 03:36:05.586991 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.586894 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-api-access-sqhjt\" (UniqueName: \"kubernetes.io/projected/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-api-access-sqhjt\") pod \"c817a3e1-558b-49f4-a49d-58989edc73bf\" (UID: \"c817a3e1-558b-49f4-a49d-58989edc73bf\") " Oct 30 03:36:05.587228 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.587204 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-rbac-proxy-sar-config" pod "c817a3e1-558b-49f4-a49d-58989edc73bf" (UID: "c817a3e1-558b-49f4-a49d-58989edc73bf"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:36:05.588898 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.588877 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-api-access-sqhjt" pod "c817a3e1-558b-49f4-a49d-58989edc73bf" (UID: "c817a3e1-558b-49f4-a49d-58989edc73bf"). InnerVolumeSpecName "kube-api-access-sqhjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:36:05.588963 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.588905 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c817a3e1-558b-49f4-a49d-58989edc73bf-proxy-tls" pod "c817a3e1-558b-49f4-a49d-58989edc73bf" (UID: "c817a3e1-558b-49f4-a49d-58989edc73bf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:36:05.688101 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.688016 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c817a3e1-558b-49f4-a49d-58989edc73bf-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:05.688101 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.688053 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:05.688101 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:05.688069 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqhjt\" (UniqueName: \"kubernetes.io/projected/c817a3e1-558b-49f4-a49d-58989edc73bf-kube-api-access-sqhjt\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:06.065032 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.065004 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:36:06.090317 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.090292 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e25b0a42-a5f5-4828-963b-693385ea1043-proxy-tls\" (UniqueName: \"kubernetes.io/secret/e25b0a42-a5f5-4828-963b-693385ea1043-proxy-tls\") pod \"e25b0a42-a5f5-4828-963b-693385ea1043\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " Oct 30 03:36:06.090445 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.090369 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e25b0a42-a5f5-4828-963b-693385ea1043-kube-api-access-47w6w\" (UniqueName: \"kubernetes.io/projected/e25b0a42-a5f5-4828-963b-693385ea1043-kube-api-access-47w6w\") pod \"e25b0a42-a5f5-4828-963b-693385ea1043\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " Oct 30 03:36:06.090445 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.090415 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e25b0a42-a5f5-4828-963b-693385ea1043-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e25b0a42-a5f5-4828-963b-693385ea1043-kube-rbac-proxy-sar-config\") pod \"e25b0a42-a5f5-4828-963b-693385ea1043\" (UID: \"e25b0a42-a5f5-4828-963b-693385ea1043\") " Oct 30 03:36:06.090764 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.090744 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e25b0a42-a5f5-4828-963b-693385ea1043-kube-rbac-proxy-sar-config" pod "e25b0a42-a5f5-4828-963b-693385ea1043" (UID: "e25b0a42-a5f5-4828-963b-693385ea1043"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:36:06.092317 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.092291 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e25b0a42-a5f5-4828-963b-693385ea1043-proxy-tls" pod "e25b0a42-a5f5-4828-963b-693385ea1043" (UID: "e25b0a42-a5f5-4828-963b-693385ea1043"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:36:06.092389 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.092329 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25b0a42-a5f5-4828-963b-693385ea1043-kube-api-access-47w6w" pod "e25b0a42-a5f5-4828-963b-693385ea1043" (UID: "e25b0a42-a5f5-4828-963b-693385ea1043"). InnerVolumeSpecName "kube-api-access-47w6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:36:06.191454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.191370 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e25b0a42-a5f5-4828-963b-693385ea1043-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:06.191454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.191399 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e25b0a42-a5f5-4828-963b-693385ea1043-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:06.191454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.191410 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47w6w\" (UniqueName: \"kubernetes.io/projected/e25b0a42-a5f5-4828-963b-693385ea1043-kube-api-access-47w6w\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:06.327052 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.327021 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" Oct 30 03:36:06.327214 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.327023 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5" event={"ID":"c817a3e1-558b-49f4-a49d-58989edc73bf","Type":"ContainerDied","Data":"bf550079b56142cfcc441765ebcba21a9f5684aa42532b67b32c898dd22f78dc"} Oct 30 03:36:06.327214 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.327200 4524 scope.go:122] "RemoveContainer" containerID="50fd136ba4a60be0454fe61620d8dcbeb4390a17971449695b5733b664d9e402" Oct 30 03:36:06.328773 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.328676 4524 generic.go:356] "Generic (PLEG): container finished" podID="e25b0a42-a5f5-4828-963b-693385ea1043" containerID="963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66" exitCode=0 Oct 30 03:36:06.328773 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.328717 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" event={"ID":"e25b0a42-a5f5-4828-963b-693385ea1043","Type":"ContainerDied","Data":"963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66"} Oct 30 03:36:06.328773 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.328731 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" Oct 30 03:36:06.328773 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.328762 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249" event={"ID":"e25b0a42-a5f5-4828-963b-693385ea1043","Type":"ContainerDied","Data":"270578552abb7723d876458b6570d9dd04099af9c896dcc0d9fc06369a6488fe"} Oct 30 03:36:06.335632 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.335613 4524 scope.go:122] "RemoveContainer" containerID="756693f824028e02d8107688481155a75e44528cf556be4c06edc9da76a1dffc" Oct 30 03:36:06.344026 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.344001 4524 scope.go:122] "RemoveContainer" containerID="01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea" Oct 30 03:36:06.350887 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.350872 4524 scope.go:122] "RemoveContainer" containerID="963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66" Oct 30 03:36:06.357259 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.357242 4524 scope.go:122] "RemoveContainer" containerID="01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea" Oct 30 03:36:06.357509 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:06.357489 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea\": container with ID starting with 01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea not found: ID does not exist" containerID="01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea" Oct 30 03:36:06.357609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.357546 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea"} err="failed to get container status \"01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea\": rpc error: code = NotFound desc = could not find container \"01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea\": container with ID starting with 01d06c77d54895a65543909bf79bf7095acf61ad748f968b3fd3cf9eab74d6ea not found: ID does not exist" Oct 30 03:36:06.357609 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.357562 4524 scope.go:122] "RemoveContainer" containerID="963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66" Oct 30 03:36:06.357812 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:06.357795 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66\": container with ID starting with 963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66 not found: ID does not exist" containerID="963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66" Oct 30 03:36:06.357857 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.357818 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66"} err="failed to get container status \"963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66\": rpc error: code = NotFound desc = could not find container \"963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66\": container with ID starting with 963e3517e938c657c66328ea5b946fc8969df95817d660e97c605013fd70de66 not found: ID does not exist" Oct 30 03:36:06.388090 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.388066 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249"] Oct 30 03:36:06.402330 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.402305 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-81fd6-predictor-5c74697485-zb249"] Oct 30 03:36:06.436287 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.436259 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5"] Oct 30 03:36:06.441775 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:06.441721 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-81fd6-predictor-79ccc896b4-v47d5"] Oct 30 03:36:07.020171 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.020138 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" path="/var/lib/kubelet/pods/c817a3e1-558b-49f4-a49d-58989edc73bf/volumes" Oct 30 03:36:07.020568 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.020553 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" path="/var/lib/kubelet/pods/e25b0a42-a5f5-4828-963b-693385ea1043/volumes" Oct 30 03:36:07.596954 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.596918 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22"] Oct 30 03:36:07.597446 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597372 4524 cpu_manager.go:414] "containerMap: removing container" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kube-rbac-proxy" Oct 30 03:36:07.597446 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597390 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kube-rbac-proxy" Oct 30 03:36:07.597446 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597412 4524 cpu_manager.go:414] "containerMap: removing container" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" Oct 30 03:36:07.597446 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597419 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" Oct 30 03:36:07.597446 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597434 4524 cpu_manager.go:414] "containerMap: removing container" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" Oct 30 03:36:07.597446 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597443 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" Oct 30 03:36:07.597792 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597454 4524 cpu_manager.go:414] "containerMap: removing container" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kube-rbac-proxy" Oct 30 03:36:07.597792 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597461 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kube-rbac-proxy" Oct 30 03:36:07.597792 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597564 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kube-rbac-proxy" Oct 30 03:36:07.597792 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597578 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e25b0a42-a5f5-4828-963b-693385ea1043" containerName="kserve-container" Oct 30 03:36:07.597792 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597589 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kserve-container" Oct 30 03:36:07.597792 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.597598 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c817a3e1-558b-49f4-a49d-58989edc73bf" containerName="kube-rbac-proxy" Oct 30 03:36:07.601890 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.601871 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.622620 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.622598 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22"] Oct 30 03:36:07.635771 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.635747 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"success-200-isvc-1be18-predictor-serving-cert\"" Oct 30 03:36:07.635905 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.635886 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"kube-root-ca.crt\"" Oct 30 03:36:07.636388 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.636373 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"openshift-service-ca.crt\"" Oct 30 03:36:07.636491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.636398 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"default-dockercfg-tlgnp\"" Oct 30 03:36:07.636491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.636404 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"success-200-isvc-1be18-kube-rbac-proxy-sar-config\"" Oct 30 03:36:07.704483 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.704454 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb60aac8-4862-44e8-a940-7c34d633c236-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.704651 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.704492 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb60aac8-4862-44e8-a940-7c34d633c236-proxy-tls\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.704651 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.704509 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd5zm\" (UniqueName: \"kubernetes.io/projected/fb60aac8-4862-44e8-a940-7c34d633c236-kube-api-access-wd5zm\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.805826 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.805787 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb60aac8-4862-44e8-a940-7c34d633c236-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.805998 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.805841 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb60aac8-4862-44e8-a940-7c34d633c236-proxy-tls\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.805998 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.805867 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd5zm\" (UniqueName: \"kubernetes.io/projected/fb60aac8-4862-44e8-a940-7c34d633c236-kube-api-access-wd5zm\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.806432 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.806409 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb60aac8-4862-44e8-a940-7c34d633c236-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.808274 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.808253 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb60aac8-4862-44e8-a940-7c34d633c236-proxy-tls\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.833544 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.833496 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd5zm\" (UniqueName: \"kubernetes.io/projected/fb60aac8-4862-44e8-a940-7c34d633c236-kube-api-access-wd5zm\") pod \"success-200-isvc-1be18-predictor-587df4f4d4-5vq22\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.916804 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.916738 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:07.982875 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.982849 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp"] Oct 30 03:36:07.987300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:07.987271 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.007879 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.007856 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"error-404-isvc-1be18-kube-rbac-proxy-sar-config\"" Oct 30 03:36:08.008047 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.008010 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a578f5e-48e3-438d-91a1-07dbbba9b31e-proxy-tls\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.008161 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.008143 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-0ae4d744aa\"/\"error-404-isvc-1be18-predictor-serving-cert\"" Oct 30 03:36:08.008239 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.008188 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.008239 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.008211 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65w4j\" (UniqueName: \"kubernetes.io/projected/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-api-access-65w4j\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.009390 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.009366 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp"] Oct 30 03:36:08.109560 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.109532 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.109718 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.109566 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65w4j\" (UniqueName: \"kubernetes.io/projected/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-api-access-65w4j\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.109718 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.109595 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a578f5e-48e3-438d-91a1-07dbbba9b31e-proxy-tls\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.110154 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.110130 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.111951 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.111935 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a578f5e-48e3-438d-91a1-07dbbba9b31e-proxy-tls\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.137041 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.137017 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65w4j\" (UniqueName: \"kubernetes.io/projected/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-api-access-65w4j\") pod \"error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.213151 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.213125 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22"] Oct 30 03:36:08.213545 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:36:08.213510 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb60aac8_4862_44e8_a940_7c34d633c236.slice/crio-0860e852a0f72e5429c5b8e55ab8847a828e9233b693d78048ce7aa142acfc7b WatchSource:0}: Error finding container 0860e852a0f72e5429c5b8e55ab8847a828e9233b693d78048ce7aa142acfc7b: Status 404 returned error can't find the container with id 0860e852a0f72e5429c5b8e55ab8847a828e9233b693d78048ce7aa142acfc7b Oct 30 03:36:08.302454 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.302425 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:08.341663 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.341630 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" event={"ID":"fb60aac8-4862-44e8-a940-7c34d633c236","Type":"ContainerStarted","Data":"1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f"} Oct 30 03:36:08.341858 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.341687 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" event={"ID":"fb60aac8-4862-44e8-a940-7c34d633c236","Type":"ContainerStarted","Data":"ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8"} Oct 30 03:36:08.341858 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.341708 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" event={"ID":"fb60aac8-4862-44e8-a940-7c34d633c236","Type":"ContainerStarted","Data":"0860e852a0f72e5429c5b8e55ab8847a828e9233b693d78048ce7aa142acfc7b"} Oct 30 03:36:08.342861 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.342650 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:08.342992 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.342884 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:08.351258 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.351221 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused" Oct 30 03:36:08.395208 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.395160 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podStartSLOduration=1.395144661 podStartE2EDuration="1.395144661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:36:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:36:08.393592079 +0000 UTC m=+1911.515158645" watchObservedRunningTime="2026-10-30 03:36:08.395144661 +0000 UTC m=+1911.516711230" Oct 30 03:36:08.601562 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:36:08.601536 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a578f5e_48e3_438d_91a1_07dbbba9b31e.slice/crio-a9b3eebf7a5e71b25e6f705d55c0f5c9f04ff4b9ad93d19dd7abffd3d4364efd WatchSource:0}: Error finding container a9b3eebf7a5e71b25e6f705d55c0f5c9f04ff4b9ad93d19dd7abffd3d4364efd: Status 404 returned error can't find the container with id a9b3eebf7a5e71b25e6f705d55c0f5c9f04ff4b9ad93d19dd7abffd3d4364efd Oct 30 03:36:08.601896 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:08.601868 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp"] Oct 30 03:36:09.348094 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:09.348055 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" event={"ID":"5a578f5e-48e3-438d-91a1-07dbbba9b31e","Type":"ContainerStarted","Data":"bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7"} Oct 30 03:36:09.348294 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:09.348272 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:09.348385 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:09.348293 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused" Oct 30 03:36:09.348385 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:09.348328 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:09.348385 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:09.348371 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" event={"ID":"5a578f5e-48e3-438d-91a1-07dbbba9b31e","Type":"ContainerStarted","Data":"840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb"} Oct 30 03:36:09.348557 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:09.348401 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" event={"ID":"5a578f5e-48e3-438d-91a1-07dbbba9b31e","Type":"ContainerStarted","Data":"a9b3eebf7a5e71b25e6f705d55c0f5c9f04ff4b9ad93d19dd7abffd3d4364efd"} Oct 30 03:36:09.349595 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:09.349572 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused" Oct 30 03:36:09.398046 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:09.397996 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podStartSLOduration=2.397984905 podStartE2EDuration="2.397984905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:36:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:36:09.397758775 +0000 UTC m=+1912.519325341" watchObservedRunningTime="2026-10-30 03:36:09.397984905 +0000 UTC m=+1912.519551472" Oct 30 03:36:10.351972 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:10.351933 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused" Oct 30 03:36:10.352327 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:10.352011 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused" Oct 30 03:36:12.160310 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:12.160270 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused" Oct 30 03:36:12.160744 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:12.160270 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused" Oct 30 03:36:15.358031 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:15.357994 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:36:15.358641 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:15.358158 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:36:15.362823 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:15.362787 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused" Oct 30 03:36:15.362970 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:15.362899 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused" Oct 30 03:36:19.337622 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:19.337588 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:36:22.161360 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:22.161331 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:36:22.161785 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:22.161432 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:36:25.358629 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:25.358590 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused" Oct 30 03:36:25.358629 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:25.358634 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused" Oct 30 03:36:35.359155 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:35.359114 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused" Oct 30 03:36:35.359548 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:35.359150 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused" Oct 30 03:36:45.359180 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:45.359136 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused" Oct 30 03:36:45.359607 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:45.359182 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused" Oct 30 03:36:46.109126 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.109094 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9"] Oct 30 03:36:46.109320 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.109292 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" containerID="cri-o://33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3" gracePeriod=30 Oct 30 03:36:46.109373 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.109327 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kube-rbac-proxy" containerID="cri-o://d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7" gracePeriod=30 Oct 30 03:36:46.204496 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.204467 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888"] Oct 30 03:36:46.204711 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.204689 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" containerID="cri-o://c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91" gracePeriod=30 Oct 30 03:36:46.204760 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.204727 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kube-rbac-proxy" containerID="cri-o://b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3" gracePeriod=30 Oct 30 03:36:46.488742 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.488660 4524 generic.go:356] "Generic (PLEG): container finished" podID="113785af-1748-4ea5-982a-b872963ffc4e" containerID="d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7" exitCode=0 Oct 30 03:36:46.488742 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.488720 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" event={"ID":"113785af-1748-4ea5-982a-b872963ffc4e","Type":"ContainerDied","Data":"d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7"} Oct 30 03:36:46.490568 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.490547 4524 generic.go:356] "Generic (PLEG): container finished" podID="4fd2a247-b042-4530-9b89-874c1ac12780" containerID="b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3" exitCode=0 Oct 30 03:36:46.490684 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:46.490592 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" event={"ID":"4fd2a247-b042-4530-9b89-874c1ac12780","Type":"ContainerDied","Data":"b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3"} Oct 30 03:36:47.154999 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:47.154952 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.39:8643/healthz\": dial tcp 10.133.0.39:8643: connect: connection refused" Oct 30 03:36:47.155174 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:47.155003 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused" Oct 30 03:36:49.257228 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:49.257197 4524 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113785af_1748_4ea5_982a_b872963ffc4e.slice/crio-33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113785af_1748_4ea5_982a_b872963ffc4e.slice/crio-conmon-33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3.scope\": RecentStats: unable to find data in memory cache]" Oct 30 03:36:49.339493 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:49.339451 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:36:49.456137 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.456114 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:36:49.504777 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.504693 4524 generic.go:356] "Generic (PLEG): container finished" podID="113785af-1748-4ea5-982a-b872963ffc4e" containerID="33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3" exitCode=0 Oct 30 03:36:49.504777 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.504755 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" event={"ID":"113785af-1748-4ea5-982a-b872963ffc4e","Type":"ContainerDied","Data":"33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3"} Oct 30 03:36:49.505001 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.504809 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" event={"ID":"113785af-1748-4ea5-982a-b872963ffc4e","Type":"ContainerDied","Data":"62e19e73cdaa950d4b09f307a5411a506924f8bb439cfae94577aa42737a6e23"} Oct 30 03:36:49.505001 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.504839 4524 scope.go:122] "RemoveContainer" containerID="d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7" Oct 30 03:36:49.505001 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.504762 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9" Oct 30 03:36:49.513499 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.513480 4524 scope.go:122] "RemoveContainer" containerID="33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3" Oct 30 03:36:49.520848 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.520827 4524 scope.go:122] "RemoveContainer" containerID="d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7" Oct 30 03:36:49.521108 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:49.521090 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7\": container with ID starting with d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7 not found: ID does not exist" containerID="d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7" Oct 30 03:36:49.521161 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.521119 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7"} err="failed to get container status \"d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7\": rpc error: code = NotFound desc = could not find container \"d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7\": container with ID starting with d89fc0730de1d00b8c9e559f5df136ca357b4ef28bcae2c2bd761a024753daf7 not found: ID does not exist" Oct 30 03:36:49.521161 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.521142 4524 scope.go:122] "RemoveContainer" containerID="33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3" Oct 30 03:36:49.521397 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:49.521382 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3\": container with ID starting with 33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3 not found: ID does not exist" containerID="33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3" Oct 30 03:36:49.521453 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.521400 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3"} err="failed to get container status \"33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3\": rpc error: code = NotFound desc = could not find container \"33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3\": container with ID starting with 33c12ae3e01ccf16d19f1e8919bbbddb2a080bb327aa6c0cdfe7bf43cf6661d3 not found: ID does not exist" Oct 30 03:36:49.630184 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.630150 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/113785af-1748-4ea5-982a-b872963ffc4e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/113785af-1748-4ea5-982a-b872963ffc4e-kube-rbac-proxy-sar-config\") pod \"113785af-1748-4ea5-982a-b872963ffc4e\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " Oct 30 03:36:49.630376 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.630234 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/113785af-1748-4ea5-982a-b872963ffc4e-kube-api-access-7m7ml\" (UniqueName: \"kubernetes.io/projected/113785af-1748-4ea5-982a-b872963ffc4e-kube-api-access-7m7ml\") pod \"113785af-1748-4ea5-982a-b872963ffc4e\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " Oct 30 03:36:49.630376 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.630344 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/113785af-1748-4ea5-982a-b872963ffc4e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/113785af-1748-4ea5-982a-b872963ffc4e-proxy-tls\") pod \"113785af-1748-4ea5-982a-b872963ffc4e\" (UID: \"113785af-1748-4ea5-982a-b872963ffc4e\") " Oct 30 03:36:49.630649 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.630618 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/113785af-1748-4ea5-982a-b872963ffc4e-kube-rbac-proxy-sar-config" pod "113785af-1748-4ea5-982a-b872963ffc4e" (UID: "113785af-1748-4ea5-982a-b872963ffc4e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:36:49.632344 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.632314 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/113785af-1748-4ea5-982a-b872963ffc4e-kube-api-access-7m7ml" pod "113785af-1748-4ea5-982a-b872963ffc4e" (UID: "113785af-1748-4ea5-982a-b872963ffc4e"). InnerVolumeSpecName "kube-api-access-7m7ml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:36:49.632737 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.632718 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113785af-1748-4ea5-982a-b872963ffc4e-proxy-tls" pod "113785af-1748-4ea5-982a-b872963ffc4e" (UID: "113785af-1748-4ea5-982a-b872963ffc4e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:36:49.731987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.731947 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/113785af-1748-4ea5-982a-b872963ffc4e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:49.731987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.731986 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m7ml\" (UniqueName: \"kubernetes.io/projected/113785af-1748-4ea5-982a-b872963ffc4e-kube-api-access-7m7ml\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:49.732166 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.732028 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113785af-1748-4ea5-982a-b872963ffc4e-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:49.857626 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.857587 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9"] Oct 30 03:36:49.871664 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:49.871637 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-32dc6-predictor-6594bd75b4-m24n9"] Oct 30 03:36:50.347413 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.347389 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:36:50.438998 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.438926 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4fd2a247-b042-4530-9b89-874c1ac12780-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd2a247-b042-4530-9b89-874c1ac12780-kube-rbac-proxy-sar-config\") pod \"4fd2a247-b042-4530-9b89-874c1ac12780\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " Oct 30 03:36:50.439136 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.439006 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4fd2a247-b042-4530-9b89-874c1ac12780-proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd2a247-b042-4530-9b89-874c1ac12780-proxy-tls\") pod \"4fd2a247-b042-4530-9b89-874c1ac12780\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " Oct 30 03:36:50.439188 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.439166 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fd2a247-b042-4530-9b89-874c1ac12780-kube-api-access-p6nj8\" (UniqueName: \"kubernetes.io/projected/4fd2a247-b042-4530-9b89-874c1ac12780-kube-api-access-p6nj8\") pod \"4fd2a247-b042-4530-9b89-874c1ac12780\" (UID: \"4fd2a247-b042-4530-9b89-874c1ac12780\") " Oct 30 03:36:50.439296 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.439277 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd2a247-b042-4530-9b89-874c1ac12780-kube-rbac-proxy-sar-config" pod "4fd2a247-b042-4530-9b89-874c1ac12780" (UID: "4fd2a247-b042-4530-9b89-874c1ac12780"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:36:50.439433 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.439420 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4fd2a247-b042-4530-9b89-874c1ac12780-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:50.440968 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.440943 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fd2a247-b042-4530-9b89-874c1ac12780-proxy-tls" pod "4fd2a247-b042-4530-9b89-874c1ac12780" (UID: "4fd2a247-b042-4530-9b89-874c1ac12780"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:36:50.441084 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.441026 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fd2a247-b042-4530-9b89-874c1ac12780-kube-api-access-p6nj8" pod "4fd2a247-b042-4530-9b89-874c1ac12780" (UID: "4fd2a247-b042-4530-9b89-874c1ac12780"). InnerVolumeSpecName "kube-api-access-p6nj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:36:50.511505 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.511476 4524 generic.go:356] "Generic (PLEG): container finished" podID="4fd2a247-b042-4530-9b89-874c1ac12780" containerID="c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91" exitCode=0 Oct 30 03:36:50.511642 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.511539 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" Oct 30 03:36:50.511642 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.511540 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" event={"ID":"4fd2a247-b042-4530-9b89-874c1ac12780","Type":"ContainerDied","Data":"c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91"} Oct 30 03:36:50.511729 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.511658 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888" event={"ID":"4fd2a247-b042-4530-9b89-874c1ac12780","Type":"ContainerDied","Data":"b629f1b3746109224d44dd9c997bc2b7efe3b99dde9104338728184a1cd0d0e2"} Oct 30 03:36:50.511729 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.511682 4524 scope.go:122] "RemoveContainer" containerID="b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3" Oct 30 03:36:50.522829 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.521155 4524 scope.go:122] "RemoveContainer" containerID="c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91" Oct 30 03:36:50.530272 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.530251 4524 scope.go:122] "RemoveContainer" containerID="b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3" Oct 30 03:36:50.530593 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:50.530575 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3\": container with ID starting with b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3 not found: ID does not exist" containerID="b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3" Oct 30 03:36:50.530655 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.530602 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3"} err="failed to get container status \"b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3\": rpc error: code = NotFound desc = could not find container \"b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3\": container with ID starting with b0e9f2b35d0a29d255ed8295fa7d9d60bedc35dfe676ba9b32805906a8944cd3 not found: ID does not exist" Oct 30 03:36:50.530655 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.530619 4524 scope.go:122] "RemoveContainer" containerID="c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91" Oct 30 03:36:50.530912 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:36:50.530890 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91\": container with ID starting with c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91 not found: ID does not exist" containerID="c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91" Oct 30 03:36:50.530962 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.530922 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91"} err="failed to get container status \"c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91\": rpc error: code = NotFound desc = could not find container \"c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91\": container with ID starting with c248a2da59382f337041f165525446e74b90d60a3ab40c9f9d2454cdff45bb91 not found: ID does not exist" Oct 30 03:36:50.540275 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.540257 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4fd2a247-b042-4530-9b89-874c1ac12780-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:50.540344 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.540279 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6nj8\" (UniqueName: \"kubernetes.io/projected/4fd2a247-b042-4530-9b89-874c1ac12780-kube-api-access-p6nj8\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:36:50.572786 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.572760 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888"] Oct 30 03:36:50.579330 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:50.579309 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-32dc6-predictor-565d78d569-jz888"] Oct 30 03:36:51.023868 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.021578 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="113785af-1748-4ea5-982a-b872963ffc4e" path="/var/lib/kubelet/pods/113785af-1748-4ea5-982a-b872963ffc4e/volumes" Oct 30 03:36:51.023868 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.022182 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" path="/var/lib/kubelet/pods/4fd2a247-b042-4530-9b89-874c1ac12780/volumes" Oct 30 03:36:51.606482 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606447 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t"] Oct 30 03:36:51.606867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606769 4524 cpu_manager.go:414] "containerMap: removing container" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kube-rbac-proxy" Oct 30 03:36:51.606867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606780 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kube-rbac-proxy" Oct 30 03:36:51.606867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606795 4524 cpu_manager.go:414] "containerMap: removing container" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" Oct 30 03:36:51.606867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606800 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" Oct 30 03:36:51.606867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606808 4524 cpu_manager.go:414] "containerMap: removing container" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kube-rbac-proxy" Oct 30 03:36:51.606867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606813 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kube-rbac-proxy" Oct 30 03:36:51.606867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606825 4524 cpu_manager.go:414] "containerMap: removing container" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" Oct 30 03:36:51.606867 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606831 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" Oct 30 03:36:51.607125 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606877 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kserve-container" Oct 30 03:36:51.607125 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606887 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kube-rbac-proxy" Oct 30 03:36:51.607125 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606895 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="113785af-1748-4ea5-982a-b872963ffc4e" containerName="kserve-container" Oct 30 03:36:51.607125 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.606900 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fd2a247-b042-4530-9b89-874c1ac12780" containerName="kube-rbac-proxy" Oct 30 03:36:51.611323 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.611307 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.642714 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.642688 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t"] Oct 30 03:36:51.644095 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.644069 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"default-dockercfg-dtsbg\"" Oct 30 03:36:51.644229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.644111 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-defc1-predictor-serving-cert\"" Oct 30 03:36:51.644229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.644136 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"openshift-service-ca.crt\"" Oct 30 03:36:51.644229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.644141 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"success-200-isvc-defc1-kube-rbac-proxy-sar-config\"" Oct 30 03:36:51.644368 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.644357 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"kube-root-ca.crt\"" Oct 30 03:36:51.749343 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.749311 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4nj4\" (UniqueName: \"kubernetes.io/projected/dd0b1750-323d-4850-a461-cae2f6155823-kube-api-access-r4nj4\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.749484 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.749364 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd0b1750-323d-4850-a461-cae2f6155823-proxy-tls\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.749484 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.749467 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd0b1750-323d-4850-a461-cae2f6155823-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.850381 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.850351 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd0b1750-323d-4850-a461-cae2f6155823-proxy-tls\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.850537 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.850394 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd0b1750-323d-4850-a461-cae2f6155823-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.850537 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.850440 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4nj4\" (UniqueName: \"kubernetes.io/projected/dd0b1750-323d-4850-a461-cae2f6155823-kube-api-access-r4nj4\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.851130 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.851109 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd0b1750-323d-4850-a461-cae2f6155823-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.852768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.852750 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd0b1750-323d-4850-a461-cae2f6155823-proxy-tls\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.878414 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.878348 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4nj4\" (UniqueName: \"kubernetes.io/projected/dd0b1750-323d-4850-a461-cae2f6155823-kube-api-access-r4nj4\") pod \"success-200-isvc-defc1-predictor-7748d8df75-ndm7t\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:51.925190 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:51.925164 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:52.003810 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.003781 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59"] Oct 30 03:36:52.007353 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.007338 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.031545 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.031499 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59"] Oct 30 03:36:52.031753 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.031735 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-defc1-kube-rbac-proxy-sar-config\"" Oct 30 03:36:52.031823 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.031797 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b5931f0e7\"/\"error-404-isvc-defc1-predictor-serving-cert\"" Oct 30 03:36:52.153069 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.152990 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8scqw\" (UniqueName: \"kubernetes.io/projected/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-api-access-8scqw\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.153069 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.153023 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-proxy-tls\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.153069 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.153041 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.223371 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:36:52.223343 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0b1750_323d_4850_a461_cae2f6155823.slice/crio-df9dea3c1c6e23f2fcae0add7d7049a07a8cef1302199fbe5fd8dc30aaebf6fc WatchSource:0}: Error finding container df9dea3c1c6e23f2fcae0add7d7049a07a8cef1302199fbe5fd8dc30aaebf6fc: Status 404 returned error can't find the container with id df9dea3c1c6e23f2fcae0add7d7049a07a8cef1302199fbe5fd8dc30aaebf6fc Oct 30 03:36:52.223499 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.223415 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t"] Oct 30 03:36:52.254390 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.254365 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8scqw\" (UniqueName: \"kubernetes.io/projected/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-api-access-8scqw\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.254491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.254397 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-proxy-tls\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.254491 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.254413 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.254965 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.254949 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.256594 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.256576 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-proxy-tls\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.282777 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.282754 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8scqw\" (UniqueName: \"kubernetes.io/projected/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-api-access-8scqw\") pod \"error-404-isvc-defc1-predictor-7b64c969f4-fdf59\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.321672 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.321653 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:52.521754 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.521673 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" event={"ID":"dd0b1750-323d-4850-a461-cae2f6155823","Type":"ContainerStarted","Data":"74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df"} Oct 30 03:36:52.521754 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.521732 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" event={"ID":"dd0b1750-323d-4850-a461-cae2f6155823","Type":"ContainerStarted","Data":"d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822"} Oct 30 03:36:52.521974 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.521762 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:52.521974 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.521785 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" event={"ID":"dd0b1750-323d-4850-a461-cae2f6155823","Type":"ContainerStarted","Data":"df9dea3c1c6e23f2fcae0add7d7049a07a8cef1302199fbe5fd8dc30aaebf6fc"} Oct 30 03:36:52.521974 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.521812 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:52.523270 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.523242 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused" Oct 30 03:36:52.572168 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.572122 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podStartSLOduration=1.572111027 podStartE2EDuration="1.572111027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:36:52.571472935 +0000 UTC m=+1955.693039501" watchObservedRunningTime="2026-10-30 03:36:52.572111027 +0000 UTC m=+1955.693677594" Oct 30 03:36:52.618936 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:52.618911 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59"] Oct 30 03:36:52.619232 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:36:52.618992 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4badfb61_fe35_4dbd_bf16_f18dafe3f00f.slice/crio-1f354b270c4843eee0953253814b8d2bdc6eb14f6bbc19032e4029ccdbc227b1 WatchSource:0}: Error finding container 1f354b270c4843eee0953253814b8d2bdc6eb14f6bbc19032e4029ccdbc227b1: Status 404 returned error can't find the container with id 1f354b270c4843eee0953253814b8d2bdc6eb14f6bbc19032e4029ccdbc227b1 Oct 30 03:36:53.529547 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:53.527806 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" event={"ID":"4badfb61-fe35-4dbd-bf16-f18dafe3f00f","Type":"ContainerStarted","Data":"62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0"} Oct 30 03:36:53.529547 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:53.527886 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" event={"ID":"4badfb61-fe35-4dbd-bf16-f18dafe3f00f","Type":"ContainerStarted","Data":"85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee"} Oct 30 03:36:53.529547 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:53.527912 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" event={"ID":"4badfb61-fe35-4dbd-bf16-f18dafe3f00f","Type":"ContainerStarted","Data":"1f354b270c4843eee0953253814b8d2bdc6eb14f6bbc19032e4029ccdbc227b1"} Oct 30 03:36:53.529547 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:53.528119 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused" Oct 30 03:36:53.529547 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:53.528458 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:53.578186 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:53.578135 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podStartSLOduration=2.578122474 podStartE2EDuration="2.578122474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:36:53.577392099 +0000 UTC m=+1956.698958663" watchObservedRunningTime="2026-10-30 03:36:53.578122474 +0000 UTC m=+1956.699689040" Oct 30 03:36:54.533584 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:54.533549 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:36:54.535207 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:54.535162 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused" Oct 30 03:36:55.358567 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:55.358527 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused" Oct 30 03:36:55.358755 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:55.358629 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused" Oct 30 03:36:55.538303 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:55.538265 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused" Oct 30 03:36:58.534973 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:58.534938 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:36:58.535689 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:36:58.535653 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused" Oct 30 03:37:00.543356 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:00.543326 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:37:00.543935 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:00.543900 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused" Oct 30 03:37:05.359689 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:05.359644 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:37:05.360081 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:05.359757 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:37:08.536056 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:08.536013 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused" Oct 30 03:37:10.544502 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:10.544463 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused" Oct 30 03:37:18.535663 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:18.535623 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused" Oct 30 03:37:19.337340 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:37:19.337305 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:37:20.544423 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:20.544380 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused" Oct 30 03:37:28.535793 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:28.535754 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused" Oct 30 03:37:30.543910 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:30.543876 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused" Oct 30 03:37:38.535569 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:38.535507 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused" Oct 30 03:37:40.544681 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:40.544643 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused" Oct 30 03:37:48.536366 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:48.536336 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:37:49.337178 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:37:49.337148 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:37:50.545177 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:37:50.545145 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:38:19.336774 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:38:19.336694 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:38:49.336892 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:38:49.336858 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:39:19.337267 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:39:19.337231 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:39:49.336199 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:39:49.336167 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:40:19.336145 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:40:19.336103 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:40:49.336331 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:40:49.336291 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:41:19.336864 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:41:19.336826 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:41:49.336659 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:41:49.336625 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:42:19.337035 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:42:19.336985 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:42:49.336949 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:42:49.336915 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:43:19.336742 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:43:19.336700 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:43:49.336622 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:43:49.336587 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:44:19.337359 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:44:19.337276 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:44:49.336980 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:44:49.336946 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:45:19.336346 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:45:19.336313 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:45:49.337020 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:45:49.336984 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:46:17.317736 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.317708 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59"] Oct 30 03:46:17.318263 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.317921 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" containerID="cri-o://85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee" gracePeriod=30 Oct 30 03:46:17.318263 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.317963 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kube-rbac-proxy" containerID="cri-o://62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0" gracePeriod=30 Oct 30 03:46:17.455467 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.455427 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t"] Oct 30 03:46:17.455849 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.455794 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" containerID="cri-o://d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822" gracePeriod=30 Oct 30 03:46:17.456297 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.456230 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kube-rbac-proxy" containerID="cri-o://74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df" gracePeriod=30 Oct 30 03:46:17.734098 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.734056 4524 generic.go:356] "Generic (PLEG): container finished" podID="dd0b1750-323d-4850-a461-cae2f6155823" containerID="74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df" exitCode=0 Oct 30 03:46:17.734098 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.734094 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" event={"ID":"dd0b1750-323d-4850-a461-cae2f6155823","Type":"ContainerDied","Data":"74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df"} Oct 30 03:46:17.736128 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.736098 4524 generic.go:356] "Generic (PLEG): container finished" podID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerID="62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0" exitCode=0 Oct 30 03:46:17.736253 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:17.736147 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" event={"ID":"4badfb61-fe35-4dbd-bf16-f18dafe3f00f","Type":"ContainerDied","Data":"62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0"} Oct 30 03:46:18.528381 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:18.528343 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.43:8643/healthz\": dial tcp 10.133.0.43:8643: connect: connection refused" Oct 30 03:46:18.535960 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:18.535931 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused" Oct 30 03:46:19.337112 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:46:19.337075 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:46:20.539378 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:20.539343 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.44:8643/healthz\": dial tcp 10.133.0.44:8643: connect: connection refused" Oct 30 03:46:20.544798 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:20.544774 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused" Oct 30 03:46:21.021884 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.021859 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:46:21.042240 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.042216 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-proxy-tls\" (UniqueName: \"kubernetes.io/secret/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-proxy-tls\") pod \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " Oct 30 03:46:21.042372 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.042267 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-api-access-8scqw\" (UniqueName: \"kubernetes.io/projected/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-api-access-8scqw\") pod \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " Oct 30 03:46:21.042372 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.042292 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-rbac-proxy-sar-config\") pod \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\" (UID: \"4badfb61-fe35-4dbd-bf16-f18dafe3f00f\") " Oct 30 03:46:21.042763 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.042729 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-rbac-proxy-sar-config" pod "4badfb61-fe35-4dbd-bf16-f18dafe3f00f" (UID: "4badfb61-fe35-4dbd-bf16-f18dafe3f00f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:46:21.044279 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.044254 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-api-access-8scqw" pod "4badfb61-fe35-4dbd-bf16-f18dafe3f00f" (UID: "4badfb61-fe35-4dbd-bf16-f18dafe3f00f"). InnerVolumeSpecName "kube-api-access-8scqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:46:21.044507 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.044494 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-proxy-tls" pod "4badfb61-fe35-4dbd-bf16-f18dafe3f00f" (UID: "4badfb61-fe35-4dbd-bf16-f18dafe3f00f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:46:21.086119 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.086070 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:46:21.143086 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.143058 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd0b1750-323d-4850-a461-cae2f6155823-kube-api-access-r4nj4\" (UniqueName: \"kubernetes.io/projected/dd0b1750-323d-4850-a461-cae2f6155823-kube-api-access-r4nj4\") pod \"dd0b1750-323d-4850-a461-cae2f6155823\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " Oct 30 03:46:21.143229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.143107 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd0b1750-323d-4850-a461-cae2f6155823-proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd0b1750-323d-4850-a461-cae2f6155823-proxy-tls\") pod \"dd0b1750-323d-4850-a461-cae2f6155823\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " Oct 30 03:46:21.143229 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.143128 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd0b1750-323d-4850-a461-cae2f6155823-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd0b1750-323d-4850-a461-cae2f6155823-kube-rbac-proxy-sar-config\") pod \"dd0b1750-323d-4850-a461-cae2f6155823\" (UID: \"dd0b1750-323d-4850-a461-cae2f6155823\") " Oct 30 03:46:21.143343 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.143309 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:46:21.143343 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.143320 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8scqw\" (UniqueName: \"kubernetes.io/projected/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-api-access-8scqw\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:46:21.143343 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.143330 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4badfb61-fe35-4dbd-bf16-f18dafe3f00f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:46:21.143561 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.143535 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd0b1750-323d-4850-a461-cae2f6155823-kube-rbac-proxy-sar-config" pod "dd0b1750-323d-4850-a461-cae2f6155823" (UID: "dd0b1750-323d-4850-a461-cae2f6155823"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:46:21.145259 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.145231 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd0b1750-323d-4850-a461-cae2f6155823-proxy-tls" pod "dd0b1750-323d-4850-a461-cae2f6155823" (UID: "dd0b1750-323d-4850-a461-cae2f6155823"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:46:21.145377 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.145351 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd0b1750-323d-4850-a461-cae2f6155823-kube-api-access-r4nj4" pod "dd0b1750-323d-4850-a461-cae2f6155823" (UID: "dd0b1750-323d-4850-a461-cae2f6155823"). InnerVolumeSpecName "kube-api-access-r4nj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:46:21.243962 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.243938 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4nj4\" (UniqueName: \"kubernetes.io/projected/dd0b1750-323d-4850-a461-cae2f6155823-kube-api-access-r4nj4\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:46:21.243962 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.243959 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd0b1750-323d-4850-a461-cae2f6155823-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:46:21.244121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.243969 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd0b1750-323d-4850-a461-cae2f6155823-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:46:21.754153 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.754123 4524 generic.go:356] "Generic (PLEG): container finished" podID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerID="85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee" exitCode=0 Oct 30 03:46:21.754507 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.754176 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" Oct 30 03:46:21.754507 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.754214 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" event={"ID":"4badfb61-fe35-4dbd-bf16-f18dafe3f00f","Type":"ContainerDied","Data":"85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee"} Oct 30 03:46:21.754507 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.754280 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59" event={"ID":"4badfb61-fe35-4dbd-bf16-f18dafe3f00f","Type":"ContainerDied","Data":"1f354b270c4843eee0953253814b8d2bdc6eb14f6bbc19032e4029ccdbc227b1"} Oct 30 03:46:21.754507 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.754322 4524 scope.go:122] "RemoveContainer" containerID="62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0" Oct 30 03:46:21.755983 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.755960 4524 generic.go:356] "Generic (PLEG): container finished" podID="dd0b1750-323d-4850-a461-cae2f6155823" containerID="d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822" exitCode=0 Oct 30 03:46:21.756090 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.755998 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" Oct 30 03:46:21.756090 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.756017 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" event={"ID":"dd0b1750-323d-4850-a461-cae2f6155823","Type":"ContainerDied","Data":"d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822"} Oct 30 03:46:21.756090 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.756064 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t" event={"ID":"dd0b1750-323d-4850-a461-cae2f6155823","Type":"ContainerDied","Data":"df9dea3c1c6e23f2fcae0add7d7049a07a8cef1302199fbe5fd8dc30aaebf6fc"} Oct 30 03:46:21.762852 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.762795 4524 scope.go:122] "RemoveContainer" containerID="85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee" Oct 30 03:46:21.770005 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.769986 4524 scope.go:122] "RemoveContainer" containerID="62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0" Oct 30 03:46:21.770258 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:46:21.770241 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0\": container with ID starting with 62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0 not found: ID does not exist" containerID="62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0" Oct 30 03:46:21.770304 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.770266 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0"} err="failed to get container status \"62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0\": rpc error: code = NotFound desc = could not find container \"62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0\": container with ID starting with 62e547622f6c66afa8ed6f10e561317ab918c0468a7f570e69a3ef8d450d5ad0 not found: ID does not exist" Oct 30 03:46:21.770304 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.770284 4524 scope.go:122] "RemoveContainer" containerID="85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee" Oct 30 03:46:21.770539 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:46:21.770502 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee\": container with ID starting with 85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee not found: ID does not exist" containerID="85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee" Oct 30 03:46:21.770617 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.770551 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee"} err="failed to get container status \"85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee\": rpc error: code = NotFound desc = could not find container \"85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee\": container with ID starting with 85160192cbc2178045f4a879d397a2bddc53ec079b81b9ed956614673ffaadee not found: ID does not exist" Oct 30 03:46:21.770617 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.770574 4524 scope.go:122] "RemoveContainer" containerID="74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df" Oct 30 03:46:21.777375 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.777357 4524 scope.go:122] "RemoveContainer" containerID="d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822" Oct 30 03:46:21.783654 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.783638 4524 scope.go:122] "RemoveContainer" containerID="74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df" Oct 30 03:46:21.783888 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:46:21.783871 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df\": container with ID starting with 74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df not found: ID does not exist" containerID="74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df" Oct 30 03:46:21.783961 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.783899 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df"} err="failed to get container status \"74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df\": rpc error: code = NotFound desc = could not find container \"74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df\": container with ID starting with 74dfcc99ebb321717e3f081af79e86c45b6ce8b37aaf33a2b558b60fba0155df not found: ID does not exist" Oct 30 03:46:21.783961 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.783921 4524 scope.go:122] "RemoveContainer" containerID="d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822" Oct 30 03:46:21.784145 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:46:21.784127 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822\": container with ID starting with d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822 not found: ID does not exist" containerID="d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822" Oct 30 03:46:21.784182 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.784151 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822"} err="failed to get container status \"d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822\": rpc error: code = NotFound desc = could not find container \"d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822\": container with ID starting with d8286b3bfee383107990be9f8ca2256d0c16b45eeeef07b3347252bf9e0c5822 not found: ID does not exist" Oct 30 03:46:21.815355 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.815333 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t"] Oct 30 03:46:21.829568 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.829547 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/success-200-isvc-defc1-predictor-7748d8df75-ndm7t"] Oct 30 03:46:21.862851 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.862830 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59"] Oct 30 03:46:21.868832 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:21.868813 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b5931f0e7/error-404-isvc-defc1-predictor-7b64c969f4-fdf59"] Oct 30 03:46:23.019465 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:23.019436 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" path="/var/lib/kubelet/pods/4badfb61-fe35-4dbd-bf16-f18dafe3f00f/volumes" Oct 30 03:46:23.019923 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:46:23.019847 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd0b1750-323d-4850-a461-cae2f6155823" path="/var/lib/kubelet/pods/dd0b1750-323d-4850-a461-cae2f6155823/volumes" Oct 30 03:46:49.336978 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:46:49.336945 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:47:19.336628 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:47:19.336593 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:47:49.335941 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:47:49.335903 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:48:19.336218 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:48:19.336178 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:48:49.336379 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:48:49.336346 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:49:19.336245 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:49:19.336208 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:49:49.337284 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:49:49.337189 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:50:19.336990 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:50:19.336909 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:50:49.336848 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:50:49.336811 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:51:19.336327 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:51:19.336290 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:51:49.336292 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:51:49.336257 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:52:19.336260 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:52:19.336227 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:52:49.336158 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:52:49.336125 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:53:19.338081 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:53:19.338006 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:53:38.688738 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:38.688705 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp"] Oct 30 03:53:38.689273 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:38.688923 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" containerID="cri-o://840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb" gracePeriod=30 Oct 30 03:53:38.689273 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:38.688981 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kube-rbac-proxy" containerID="cri-o://bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7" gracePeriod=30 Oct 30 03:53:38.841324 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:38.841296 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22"] Oct 30 03:53:38.841558 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:38.841511 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" containerID="cri-o://ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8" gracePeriod=30 Oct 30 03:53:38.841625 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:38.841591 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kube-rbac-proxy" containerID="cri-o://1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f" gracePeriod=30 Oct 30 03:53:39.308197 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:39.308160 4524 generic.go:356] "Generic (PLEG): container finished" podID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerID="bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7" exitCode=0 Oct 30 03:53:39.308379 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:39.308230 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" event={"ID":"5a578f5e-48e3-438d-91a1-07dbbba9b31e","Type":"ContainerDied","Data":"bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7"} Oct 30 03:53:39.310154 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:39.310127 4524 generic.go:356] "Generic (PLEG): container finished" podID="fb60aac8-4862-44e8-a940-7c34d633c236" containerID="1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f" exitCode=0 Oct 30 03:53:39.310284 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:39.310167 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" event={"ID":"fb60aac8-4862-44e8-a940-7c34d633c236","Type":"ContainerDied","Data":"1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f"} Oct 30 03:53:40.352928 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:40.352887 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.42:8643/healthz\": dial tcp 10.133.0.42:8643: connect: connection refused" Oct 30 03:53:40.353339 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:40.352897 4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.41:8643/healthz\": dial tcp 10.133.0.41:8643: connect: connection refused" Oct 30 03:53:41.918277 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:41.918255 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:53:42.043408 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.043382 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-api-access-65w4j\" (UniqueName: \"kubernetes.io/projected/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-api-access-65w4j\") pod \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " Oct 30 03:53:42.043544 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.043438 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5a578f5e-48e3-438d-91a1-07dbbba9b31e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a578f5e-48e3-438d-91a1-07dbbba9b31e-proxy-tls\") pod \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " Oct 30 03:53:42.043544 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.043471 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-rbac-proxy-sar-config\") pod \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\" (UID: \"5a578f5e-48e3-438d-91a1-07dbbba9b31e\") " Oct 30 03:53:42.043875 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.043853 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-rbac-proxy-sar-config" pod "5a578f5e-48e3-438d-91a1-07dbbba9b31e" (UID: "5a578f5e-48e3-438d-91a1-07dbbba9b31e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:53:42.046445 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.046421 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a578f5e-48e3-438d-91a1-07dbbba9b31e-proxy-tls" pod "5a578f5e-48e3-438d-91a1-07dbbba9b31e" (UID: "5a578f5e-48e3-438d-91a1-07dbbba9b31e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:53:42.049009 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.048984 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-api-access-65w4j" pod "5a578f5e-48e3-438d-91a1-07dbbba9b31e" (UID: "5a578f5e-48e3-438d-91a1-07dbbba9b31e"). InnerVolumeSpecName "kube-api-access-65w4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:53:42.074425 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.074407 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:53:42.144602 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.144580 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65w4j\" (UniqueName: \"kubernetes.io/projected/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-api-access-65w4j\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:53:42.144713 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.144605 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a578f5e-48e3-438d-91a1-07dbbba9b31e-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:53:42.144713 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.144619 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a578f5e-48e3-438d-91a1-07dbbba9b31e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:53:42.244795 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.244772 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb60aac8-4862-44e8-a940-7c34d633c236-kube-api-access-wd5zm\" (UniqueName: \"kubernetes.io/projected/fb60aac8-4862-44e8-a940-7c34d633c236-kube-api-access-wd5zm\") pod \"fb60aac8-4862-44e8-a940-7c34d633c236\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " Oct 30 03:53:42.244909 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.244827 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb60aac8-4862-44e8-a940-7c34d633c236-proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb60aac8-4862-44e8-a940-7c34d633c236-proxy-tls\") pod \"fb60aac8-4862-44e8-a940-7c34d633c236\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " Oct 30 03:53:42.244909 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.244892 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb60aac8-4862-44e8-a940-7c34d633c236-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb60aac8-4862-44e8-a940-7c34d633c236-kube-rbac-proxy-sar-config\") pod \"fb60aac8-4862-44e8-a940-7c34d633c236\" (UID: \"fb60aac8-4862-44e8-a940-7c34d633c236\") " Oct 30 03:53:42.245292 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.245268 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb60aac8-4862-44e8-a940-7c34d633c236-kube-rbac-proxy-sar-config" pod "fb60aac8-4862-44e8-a940-7c34d633c236" (UID: "fb60aac8-4862-44e8-a940-7c34d633c236"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 03:53:42.246771 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.246750 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb60aac8-4862-44e8-a940-7c34d633c236-kube-api-access-wd5zm" pod "fb60aac8-4862-44e8-a940-7c34d633c236" (UID: "fb60aac8-4862-44e8-a940-7c34d633c236"). InnerVolumeSpecName "kube-api-access-wd5zm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:53:42.246891 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.246871 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb60aac8-4862-44e8-a940-7c34d633c236-proxy-tls" pod "fb60aac8-4862-44e8-a940-7c34d633c236" (UID: "fb60aac8-4862-44e8-a940-7c34d633c236"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 03:53:42.323188 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.323160 4524 generic.go:356] "Generic (PLEG): container finished" podID="fb60aac8-4862-44e8-a940-7c34d633c236" containerID="ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8" exitCode=0 Oct 30 03:53:42.323287 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.323214 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" Oct 30 03:53:42.323287 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.323246 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" event={"ID":"fb60aac8-4862-44e8-a940-7c34d633c236","Type":"ContainerDied","Data":"ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8"} Oct 30 03:53:42.323385 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.323316 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22" event={"ID":"fb60aac8-4862-44e8-a940-7c34d633c236","Type":"ContainerDied","Data":"0860e852a0f72e5429c5b8e55ab8847a828e9233b693d78048ce7aa142acfc7b"} Oct 30 03:53:42.323385 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.323346 4524 scope.go:122] "RemoveContainer" containerID="1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f" Oct 30 03:53:42.324912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.324893 4524 generic.go:356] "Generic (PLEG): container finished" podID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerID="840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb" exitCode=0 Oct 30 03:53:42.324987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.324924 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" event={"ID":"5a578f5e-48e3-438d-91a1-07dbbba9b31e","Type":"ContainerDied","Data":"840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb"} Oct 30 03:53:42.324987 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.324949 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" Oct 30 03:53:42.325063 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.324986 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp" event={"ID":"5a578f5e-48e3-438d-91a1-07dbbba9b31e","Type":"ContainerDied","Data":"a9b3eebf7a5e71b25e6f705d55c0f5c9f04ff4b9ad93d19dd7abffd3d4364efd"} Oct 30 03:53:42.332478 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.332459 4524 scope.go:122] "RemoveContainer" containerID="ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8" Oct 30 03:53:42.339702 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.339683 4524 scope.go:122] "RemoveContainer" containerID="1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f" Oct 30 03:53:42.339951 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:53:42.339933 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f\": container with ID starting with 1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f not found: ID does not exist" containerID="1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f" Oct 30 03:53:42.340000 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.339960 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f"} err="failed to get container status \"1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f\": rpc error: code = NotFound desc = could not find container \"1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f\": container with ID starting with 1b8126ff935e952a6276680ce0258951c03a908e431da96d7746ebac7bc9fd8f not found: ID does not exist" Oct 30 03:53:42.340000 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.339975 4524 scope.go:122] "RemoveContainer" containerID="ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8" Oct 30 03:53:42.340160 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:53:42.340145 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8\": container with ID starting with ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8 not found: ID does not exist" containerID="ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8" Oct 30 03:53:42.340201 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.340164 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8"} err="failed to get container status \"ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8\": rpc error: code = NotFound desc = could not find container \"ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8\": container with ID starting with ead73ec815c2b46281a25ece04e38e461e4c0c0ec771e466b3688e8d2045b9b8 not found: ID does not exist" Oct 30 03:53:42.340201 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.340175 4524 scope.go:122] "RemoveContainer" containerID="bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7" Oct 30 03:53:42.345527 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.345493 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd5zm\" (UniqueName: \"kubernetes.io/projected/fb60aac8-4862-44e8-a940-7c34d633c236-kube-api-access-wd5zm\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:53:42.345687 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.345533 4524 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb60aac8-4862-44e8-a940-7c34d633c236-proxy-tls\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:53:42.345687 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.345549 4524 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb60aac8-4862-44e8-a940-7c34d633c236-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:53:42.346558 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.346544 4524 scope.go:122] "RemoveContainer" containerID="840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb" Oct 30 03:53:42.352887 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.352873 4524 scope.go:122] "RemoveContainer" containerID="bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7" Oct 30 03:53:42.353129 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:53:42.353114 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7\": container with ID starting with bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7 not found: ID does not exist" containerID="bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7" Oct 30 03:53:42.353176 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.353137 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7"} err="failed to get container status \"bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7\": rpc error: code = NotFound desc = could not find container \"bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7\": container with ID starting with bb66f1c7c834681eb93ffde44cd7888286598235b08768326e0c864c56c015a7 not found: ID does not exist" Oct 30 03:53:42.353176 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.353156 4524 scope.go:122] "RemoveContainer" containerID="840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb" Oct 30 03:53:42.353363 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:53:42.353346 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb\": container with ID starting with 840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb not found: ID does not exist" containerID="840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb" Oct 30 03:53:42.353404 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.353371 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb"} err="failed to get container status \"840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb\": rpc error: code = NotFound desc = could not find container \"840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb\": container with ID starting with 840372587d2a5bde3a51e8cbfe62ee97dc448aaa882e9881d10a47ed01fcb7cb not found: ID does not exist" Oct 30 03:53:42.384557 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.384536 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22"] Oct 30 03:53:42.390726 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.390702 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-0ae4d744aa/success-200-isvc-1be18-predictor-587df4f4d4-5vq22"] Oct 30 03:53:42.432363 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.432337 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp"] Oct 30 03:53:42.438992 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:42.438969 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-0ae4d744aa/error-404-isvc-1be18-predictor-7486cdbc8f-s2kpp"] Oct 30 03:53:43.019300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:43.019233 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" path="/var/lib/kubelet/pods/5a578f5e-48e3-438d-91a1-07dbbba9b31e/volumes" Oct 30 03:53:43.019676 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:43.019637 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" path="/var/lib/kubelet/pods/fb60aac8-4862-44e8-a940-7c34d633c236/volumes" Oct 30 03:53:45.887281 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887248 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bjj2b/must-gather-m6kzl"] Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887532 4524 cpu_manager.go:414] "containerMap: removing container" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887544 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887557 4524 cpu_manager.go:414] "containerMap: removing container" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887562 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887577 4524 cpu_manager.go:414] "containerMap: removing container" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kube-rbac-proxy" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887582 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kube-rbac-proxy" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887591 4524 cpu_manager.go:414] "containerMap: removing container" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kube-rbac-proxy" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887596 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kube-rbac-proxy" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887602 4524 cpu_manager.go:414] "containerMap: removing container" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kube-rbac-proxy" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887606 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kube-rbac-proxy" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887614 4524 cpu_manager.go:414] "containerMap: removing container" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kube-rbac-proxy" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887620 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kube-rbac-proxy" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887629 4524 cpu_manager.go:414] "containerMap: removing container" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887633 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887639 4524 cpu_manager.go:414] "containerMap: removing container" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" Oct 30 03:53:45.887670 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887643 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" Oct 30 03:53:45.888121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887685 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kserve-container" Oct 30 03:53:45.888121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887693 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kserve-container" Oct 30 03:53:45.888121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887700 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kserve-container" Oct 30 03:53:45.888121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887705 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kserve-container" Oct 30 03:53:45.888121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887711 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a578f5e-48e3-438d-91a1-07dbbba9b31e" containerName="kube-rbac-proxy" Oct 30 03:53:45.888121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887719 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb60aac8-4862-44e8-a940-7c34d633c236" containerName="kube-rbac-proxy" Oct 30 03:53:45.888121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887724 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4badfb61-fe35-4dbd-bf16-f18dafe3f00f" containerName="kube-rbac-proxy" Oct 30 03:53:45.888121 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.887729 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd0b1750-323d-4850-a461-cae2f6155823" containerName="kube-rbac-proxy" Oct 30 03:53:45.891800 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.891783 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:53:45.910947 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.910917 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bjj2b/must-gather-m6kzl"] Oct 30 03:53:45.923757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.923732 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bjj2b\"/\"default-dockercfg-cfmwq\"" Oct 30 03:53:45.923757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.923735 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bjj2b\"/\"openshift-service-ca.crt\"" Oct 30 03:53:45.923757 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:45.923732 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bjj2b\"/\"kube-root-ca.crt\"" Oct 30 03:53:46.075418 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.075371 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vznt\" (UniqueName: \"kubernetes.io/projected/370283ee-3de6-4b7a-b7aa-fae3a7d25998-kube-api-access-6vznt\") pod \"must-gather-m6kzl\" (UID: \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\") " pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:53:46.075626 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.075440 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/370283ee-3de6-4b7a-b7aa-fae3a7d25998-must-gather-output\") pod \"must-gather-m6kzl\" (UID: \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\") " pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:53:46.176780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.176700 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/370283ee-3de6-4b7a-b7aa-fae3a7d25998-must-gather-output\") pod \"must-gather-m6kzl\" (UID: \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\") " pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:53:46.176780 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.176759 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vznt\" (UniqueName: \"kubernetes.io/projected/370283ee-3de6-4b7a-b7aa-fae3a7d25998-kube-api-access-6vznt\") pod \"must-gather-m6kzl\" (UID: \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\") " pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:53:46.177081 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.177059 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/370283ee-3de6-4b7a-b7aa-fae3a7d25998-must-gather-output\") pod \"must-gather-m6kzl\" (UID: \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\") " pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:53:46.204355 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.204320 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vznt\" (UniqueName: \"kubernetes.io/projected/370283ee-3de6-4b7a-b7aa-fae3a7d25998-kube-api-access-6vznt\") pod \"must-gather-m6kzl\" (UID: \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\") " pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:53:46.217000 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.216977 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:53:46.512469 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.512443 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bjj2b/must-gather-m6kzl"] Oct 30 03:53:46.513777 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:53:46.513748 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod370283ee_3de6_4b7a_b7aa_fae3a7d25998.slice/crio-d572aca59e822a575b311984a94d737f6c3c44cf737d0aae1af6ca1829cb632b WatchSource:0}: Error finding container d572aca59e822a575b311984a94d737f6c3c44cf737d0aae1af6ca1829cb632b: Status 404 returned error can't find the container with id d572aca59e822a575b311984a94d737f6c3c44cf737d0aae1af6ca1829cb632b Oct 30 03:53:46.515639 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:46.515617 4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 03:53:47.349602 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:47.349562 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" event={"ID":"370283ee-3de6-4b7a-b7aa-fae3a7d25998","Type":"ContainerStarted","Data":"d572aca59e822a575b311984a94d737f6c3c44cf737d0aae1af6ca1829cb632b"} Oct 30 03:53:49.340459 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:53:49.340420 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:53:53.381507 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:53.381465 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" event={"ID":"370283ee-3de6-4b7a-b7aa-fae3a7d25998","Type":"ContainerStarted","Data":"717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f"} Oct 30 03:53:53.381507 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:53.381511 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" event={"ID":"370283ee-3de6-4b7a-b7aa-fae3a7d25998","Type":"ContainerStarted","Data":"15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574"} Oct 30 03:53:53.428375 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:53:53.428322 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" podStartSLOduration=2.481771761 podStartE2EDuration="8.428308124s" totalImagesPullingTime="5.946536363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 03:53:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:53:53.427337727 +0000 UTC m=+2976.548904293" watchObservedRunningTime="2026-10-30 03:53:53.428308124 +0000 UTC m=+2976.549874690" Oct 30 03:54:19.337406 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:19.337371 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:54:49.337006 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:49.336972 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:54:49.591208 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:49.591174 4524 generic.go:356] "Generic (PLEG): container finished" podID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerID="15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574" exitCode=0 Oct 30 03:54:49.593338 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:49.591247 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" event={"ID":"370283ee-3de6-4b7a-b7aa-fae3a7d25998","Type":"ContainerDied","Data":"15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574"} Oct 30 03:54:49.593338 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:49.591588 4524 scope.go:122] "RemoveContainer" containerID="15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574" Oct 30 03:54:50.257545 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:50.257505 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bjj2b_must-gather-m6kzl_370283ee-3de6-4b7a-b7aa-fae3a7d25998/gather/0.log" Oct 30 03:54:55.623251 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:55.623213 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2p9zt_1bac3d0d-dded-4047-81d5-333e47612f93/global-pull-secret-syncer/0.log" Oct 30 03:54:56.040795 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.040716 4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-bjj2b/must-gather-m6kzl"] Oct 30 03:54:56.040953 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.040896 4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerName="copy" containerID="cri-o://717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f" gracePeriod=2 Oct 30 03:54:56.056339 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.056311 4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-bjj2b/must-gather-m6kzl"] Oct 30 03:54:56.056468 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:56.056443 4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-m6kzl\" is forbidden: User \"system:node:ip-10-0-132-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bjj2b\": no relationship found between node 'ip-10-0-132-171.ec2.internal' and this object" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:54:56.274694 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.274675 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bjj2b_must-gather-m6kzl_370283ee-3de6-4b7a-b7aa-fae3a7d25998/copy/0.log" Oct 30 03:54:56.277194 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.277179 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:54:56.292663 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:56.292612 4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-m6kzl\" is forbidden: User \"system:node:ip-10-0-132-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bjj2b\": no relationship found between node 'ip-10-0-132-171.ec2.internal' and this object" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:54:56.357924 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.357901 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/370283ee-3de6-4b7a-b7aa-fae3a7d25998-kube-api-access-6vznt\" (UniqueName: \"kubernetes.io/projected/370283ee-3de6-4b7a-b7aa-fae3a7d25998-kube-api-access-6vznt\") pod \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\" (UID: \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\") " Oct 30 03:54:56.358014 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.357940 4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370283ee-3de6-4b7a-b7aa-fae3a7d25998-must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/370283ee-3de6-4b7a-b7aa-fae3a7d25998-must-gather-output\") pod \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\" (UID: \"370283ee-3de6-4b7a-b7aa-fae3a7d25998\") " Oct 30 03:54:56.359962 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.359940 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/370283ee-3de6-4b7a-b7aa-fae3a7d25998-kube-api-access-6vznt" pod "370283ee-3de6-4b7a-b7aa-fae3a7d25998" (UID: "370283ee-3de6-4b7a-b7aa-fae3a7d25998"). InnerVolumeSpecName "kube-api-access-6vznt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 03:54:56.361301 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.361275 4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370283ee-3de6-4b7a-b7aa-fae3a7d25998-must-gather-output" pod "370283ee-3de6-4b7a-b7aa-fae3a7d25998" (UID: "370283ee-3de6-4b7a-b7aa-fae3a7d25998"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 03:54:56.459133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.459109 4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vznt\" (UniqueName: \"kubernetes.io/projected/370283ee-3de6-4b7a-b7aa-fae3a7d25998-kube-api-access-6vznt\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:54:56.459133 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.459131 4524 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/370283ee-3de6-4b7a-b7aa-fae3a7d25998-must-gather-output\") on node \"ip-10-0-132-171.ec2.internal\" DevicePath \"\"" Oct 30 03:54:56.615256 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.615229 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bjj2b_must-gather-m6kzl_370283ee-3de6-4b7a-b7aa-fae3a7d25998/copy/0.log" Oct 30 03:54:56.615581 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.615556 4524 generic.go:356] "Generic (PLEG): container finished" podID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerID="717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f" exitCode=143 Oct 30 03:54:56.615692 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.615613 4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:54:56.615692 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.615642 4524 scope.go:122] "RemoveContainer" containerID="717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f" Oct 30 03:54:56.623044 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.623029 4524 scope.go:122] "RemoveContainer" containerID="15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574" Oct 30 03:54:56.631214 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:56.631175 4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-m6kzl\" is forbidden: User \"system:node:ip-10-0-132-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bjj2b\": no relationship found between node 'ip-10-0-132-171.ec2.internal' and this object" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:54:56.635696 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.635677 4524 scope.go:122] "RemoveContainer" containerID="717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f" Oct 30 03:54:56.635971 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:56.635948 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f\": container with ID starting with 717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f not found: ID does not exist" containerID="717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f" Oct 30 03:54:56.636059 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.635978 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f"} err="failed to get container status \"717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f\": rpc error: code = NotFound desc = could not find container \"717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f\": container with ID starting with 717a76b775948a6e1004823480414c64bbdaad69c0f164aa593f6943ab7afd2f not found: ID does not exist" Oct 30 03:54:56.636059 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.635995 4524 scope.go:122] "RemoveContainer" containerID="15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574" Oct 30 03:54:56.636262 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:56.636247 4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574\": container with ID starting with 15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574 not found: ID does not exist" containerID="15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574" Oct 30 03:54:56.636312 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.636265 4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574"} err="failed to get container status \"15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574\": rpc error: code = NotFound desc = could not find container \"15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574\": container with ID starting with 15588b315d6fe749d4f1b19feec2e298e5f66a2e6ee2aaf0965ae843f34ce574 not found: ID does not exist" Oct 30 03:54:56.646548 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:56.646508 4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-m6kzl\" is forbidden: User \"system:node:ip-10-0-132-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bjj2b\": no relationship found between node 'ip-10-0-132-171.ec2.internal' and this object" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:54:56.747098 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.747078 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-plrrl_9a093097-53a0-40a5-9152-0e2930424cec/konnectivity-agent/0.log" Oct 30 03:54:56.870305 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:56.870234 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-171.ec2.internal_0e5cbe062ddc538d97b4fdc348b9dc76/haproxy/0.log" Oct 30 03:54:57.019963 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:54:57.019931 4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" path="/var/lib/kubelet/pods/370283ee-3de6-4b7a-b7aa-fae3a7d25998/volumes" Oct 30 03:54:57.031833 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:54:57.031805 4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-m6kzl\" is forbidden: User \"system:node:ip-10-0-132-171.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bjj2b\": no relationship found between node 'ip-10-0-132-171.ec2.internal' and this object" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" pod="openshift-must-gather-bjj2b/must-gather-m6kzl" Oct 30 03:55:04.530150 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.530111 4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c"] Oct 30 03:55:04.530768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.530532 4524 cpu_manager.go:414] "containerMap: removing container" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerName="gather" Oct 30 03:55:04.530768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.530550 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerName="gather" Oct 30 03:55:04.530768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.530571 4524 cpu_manager.go:414] "containerMap: removing container" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerName="copy" Oct 30 03:55:04.530768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.530579 4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerName="copy" Oct 30 03:55:04.530768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.530663 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerName="gather" Oct 30 03:55:04.530768 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.530675 4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="370283ee-3de6-4b7a-b7aa-fae3a7d25998" containerName="copy" Oct 30 03:55:04.535615 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.535594 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.553448 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.553426 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c"] Oct 30 03:55:04.566764 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.566743 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4c26k\"/\"kube-root-ca.crt\"" Oct 30 03:55:04.566764 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.566754 4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4c26k\"/\"openshift-service-ca.crt\"" Oct 30 03:55:04.566922 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.566779 4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4c26k\"/\"default-dockercfg-hwhcl\"" Oct 30 03:55:04.619389 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.619363 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jrzvm_56b4051c-8b50-412e-bb8b-9a4ef725081e/node-exporter/0.log" Oct 30 03:55:04.719564 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.719510 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-lib-modules\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.719731 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.719579 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-podres\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.719731 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.719690 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kzcs\" (UniqueName: \"kubernetes.io/projected/d9e7754a-e052-4656-86ac-625da152ea09-kube-api-access-7kzcs\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.719808 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.719733 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-sys\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.719843 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.719807 4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-proc\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.744467 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.744444 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jrzvm_56b4051c-8b50-412e-bb8b-9a4ef725081e/kube-rbac-proxy/0.log" Oct 30 03:55:04.820300 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820275 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-sys\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.820480 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820308 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-proc\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.820480 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820336 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-lib-modules\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.820480 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820362 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-podres\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.820480 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820396 4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kzcs\" (UniqueName: \"kubernetes.io/projected/d9e7754a-e052-4656-86ac-625da152ea09-kube-api-access-7kzcs\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.820480 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820409 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-sys\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.820480 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820409 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-proc\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.820480 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820445 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-lib-modules\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.820480 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.820448 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d9e7754a-e052-4656-86ac-625da152ea09-podres\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.848005 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.847978 4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kzcs\" (UniqueName: \"kubernetes.io/projected/d9e7754a-e052-4656-86ac-625da152ea09-kube-api-access-7kzcs\") pod \"perf-node-gather-daemonset-wl78c\" (UID: \"d9e7754a-e052-4656-86ac-625da152ea09\") " pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:04.869915 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:04.869892 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jrzvm_56b4051c-8b50-412e-bb8b-9a4ef725081e/init-textfile/0.log" Oct 30 03:55:05.146908 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:05.146835 4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:05.438619 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:05.438592 4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c"] Oct 30 03:55:05.439211 ip-10-0-132-171 kubenswrapper[4524]: W1030 03:55:05.439188 4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd9e7754a_e052_4656_86ac_625da152ea09.slice/crio-927824e80cf67a062ae8d615cd91b28708d5ae363f773223b67b73945be75447 WatchSource:0}: Error finding container 927824e80cf67a062ae8d615cd91b28708d5ae363f773223b67b73945be75447: Status 404 returned error can't find the container with id 927824e80cf67a062ae8d615cd91b28708d5ae363f773223b67b73945be75447 Oct 30 03:55:05.646686 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:05.646645 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" event={"ID":"d9e7754a-e052-4656-86ac-625da152ea09","Type":"ContainerStarted","Data":"14f8cd4b29e9a00b797ba81e50fefc6bad23fef552096c04f6ba83eb401bf384"} Oct 30 03:55:05.647104 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:05.646722 4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" event={"ID":"d9e7754a-e052-4656-86ac-625da152ea09","Type":"ContainerStarted","Data":"927824e80cf67a062ae8d615cd91b28708d5ae363f773223b67b73945be75447"} Oct 30 03:55:05.647104 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:05.646791 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:05.693805 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:05.693719 4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" podStartSLOduration=1.693707754 podStartE2EDuration="1.693707754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 03:55:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 03:55:05.69351171 +0000 UTC m=+3048.815078277" watchObservedRunningTime="2026-10-30 03:55:05.693707754 +0000 UTC m=+3048.815274320" Oct 30 03:55:06.702245 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:06.702212 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-sr2lk_efba1548-3e1c-403c-a9cf-d6f27b5b1a48/prometheus-operator/0.log" Oct 30 03:55:06.826070 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:06.826041 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-sr2lk_efba1548-3e1c-403c-a9cf-d6f27b5b1a48/kube-rbac-proxy/0.log" Oct 30 03:55:06.955412 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:06.955332 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-jx2rc_ed6376ac-50c3-4d9a-9fcc-707d164d6d75/prometheus-operator-admission-webhook/0.log" Oct 30 03:55:11.593912 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:11.593882 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-567b98f465-tjqxp_cf1e9f02-3ec0-42c3-939b-8e1017da6c47/console/0.log" Oct 30 03:55:11.659877 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:11.659848 4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4c26k/perf-node-gather-daemonset-wl78c" Oct 30 03:55:11.739338 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:11.739316 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-ctqv4_44f95e92-057f-473e-beeb-641cabd274ff/download-server/0.log" Oct 30 03:55:15.758783 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:15.758750 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nxn6c_d5170510-91aa-451e-bc7d-e18c3aaca68b/dns/0.log" Oct 30 03:55:15.897466 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:15.897433 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nxn6c_d5170510-91aa-451e-bc7d-e18c3aaca68b/kube-rbac-proxy/0.log" Oct 30 03:55:16.333777 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:16.333750 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tslzq_31deab60-7ba7-431c-9fdd-f15388845315/dns-node-resolver/0.log" Oct 30 03:55:17.477837 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:17.477810 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6dd58cfc4-bczz9_c1bd1743-b556-480a-a06d-9ee41a24c6d4/registry/0.log" Oct 30 03:55:17.617330 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:17.617245 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5pk5f_206d6d3e-a3c9-44ff-80b7-bbc97ab9fc45/node-ca/0.log" Oct 30 03:55:19.338715 ip-10-0-132-171 kubenswrapper[4524]: E1030 03:55:19.338678 4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d6fwr" containerName="extractor" Oct 30 03:55:20.832124 ip-10-0-132-171 kubenswrapper[4524]: I1030 03:55:20.832092 4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b4nrx_bdb6477c-e16b-451c-84bd-8e62d3efc79f/serve-healthcheck-canary/0.log"