Jul 01 20:46:54.255878 ip-10-0-130-150 systemd[1]: Starting Kubernetes Kubelet... Jul 01 20:46:54.666541 ip-10-0-130-150 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 20:46:54.666541 ip-10-0-130-150 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 20:46:54.666541 ip-10-0-130-150 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 20:46:54.666541 ip-10-0-130-150 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 20:46:54.666541 ip-10-0-130-150 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 20:46:54.668350 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.668240 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 20:46:54.672515 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672498 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 20:46:54.672515 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672515 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672519 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672523 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672526 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672529 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672531 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672534 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672536 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672539 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672541 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672544 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672546 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672550 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672552 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672555 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672557 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672560 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672563 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672566 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672569 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 01 20:46:54.672580 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672571 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672574 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672577 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672581 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672586 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672589 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672592 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672595 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672598 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672600 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672603 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672605 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672607 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672625 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672628 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672630 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672639 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672642 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672644 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672647 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 20:46:54.673119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672649 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672651 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672654 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672656 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672659 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672661 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672664 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672666 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672669 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672671 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672675 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672680 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672683 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672685 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672689 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672693 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672696 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672698 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672701 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672703 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 20:46:54.673599 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672706 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672708 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672710 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672713 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672715 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672718 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672720 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672723 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672725 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672728 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672730 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672732 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672735 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672738 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672740 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672743 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672745 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672748 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672750 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672753 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 20:46:54.674135 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672755 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672758 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672760 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672763 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672765 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672768 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672773 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672776 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672779 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672782 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672785 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672787 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672790 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672792 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672795 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672797 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672799 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.672802 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673515 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 20:46:54.674854 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673522 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673526 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673529 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673532 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673535 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673537 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673540 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673544 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673548 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673550 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673553 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673556 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673559 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673562 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673565 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673568 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673571 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673574 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673577 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673581 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 20:46:54.675581 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673583 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673586 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673588 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673591 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673594 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673596 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673598 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673601 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673604 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673606 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673609 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673627 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673630 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673633 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673636 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673638 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673641 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673643 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673646 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673649 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 20:46:54.676249 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673651 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673654 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673657 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673659 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673661 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673664 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673666 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673669 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673671 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673673 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673677 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673680 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673682 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673685 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673688 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673690 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673693 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673695 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673698 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 20:46:54.676778 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673700 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673702 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673705 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673707 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673709 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673712 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673715 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673717 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673720 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673723 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673726 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673729 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673731 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673734 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673736 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673738 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673740 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673743 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673745 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673748 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 20:46:54.677284 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673750 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673753 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673755 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673757 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673761 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673764 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673766 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673768 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673771 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673773 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673777 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673780 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673783 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673785 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673788 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673791 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673794 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673798 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.673801 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 20:46:54.677782 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675058 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675083 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675088 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675094 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675099 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675103 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675107 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675112 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675121 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675129 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675133 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675138 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675143 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675153 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675157 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675161 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675165 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675170 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675174 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 20:46:54.678235 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675178 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675183 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675187 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675191 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675196 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675200 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675204 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675213 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675217 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675224 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675230 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675234 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675239 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675243 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675257 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675262 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675266 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675272 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675276 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675285 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 20:46:54.678718 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675290 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675294 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675301 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675307 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675312 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675316 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675321 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675325 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675332 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675336 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675350 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675359 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675363 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675368 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675372 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675376 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675381 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675386 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675393 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 20:46:54.679204 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675397 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675402 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675406 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675410 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675419 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675424 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675429 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675434 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675438 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675443 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675447 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675451 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675456 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675460 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675468 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675479 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675483 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675488 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675492 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 20:46:54.679680 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675497 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675501 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675505 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675509 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675514 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675518 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675522 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675526 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675531 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675541 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675545 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675549 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675555 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675559 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675563 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675567 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675572 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675576 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675581 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675585 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 20:46:54.680144 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675590 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.675594 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675770 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675864 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675932 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675941 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675951 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675957 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675964 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675972 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675977 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675990 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.675996 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676001 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676026 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676036 2558 flags.go:64] FLAG: --cgroup-root="" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676040 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676044 2558 flags.go:64] FLAG: --client-ca-file="" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676047 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676051 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676058 2558 flags.go:64] FLAG: --cluster-domain="" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676062 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676065 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676072 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 20:46:54.680636 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676076 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676116 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676227 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676236 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676240 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676243 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676246 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676249 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676252 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676256 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676261 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676269 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676272 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676275 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676278 2558 flags.go:64] FLAG: --enable-server="true" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676281 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676287 2558 flags.go:64] FLAG: --event-burst="100" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676290 2558 flags.go:64] FLAG: --event-qps="50" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676293 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676296 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676299 2558 flags.go:64] FLAG: --eviction-hard="" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676302 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676305 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676308 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676311 2558 flags.go:64] FLAG: --eviction-soft="" Jul 01 20:46:54.681210 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676313 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676316 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676319 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676322 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676325 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676328 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676331 2558 flags.go:64] FLAG: --feature-gates="" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676335 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676338 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676341 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676344 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676347 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676350 2558 flags.go:64] FLAG: --help="false" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676353 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676356 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676359 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676362 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676366 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676371 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676374 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676376 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676379 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676383 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 20:46:54.681834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676385 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676388 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676391 2558 flags.go:64] FLAG: --kube-reserved="" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676394 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676396 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676400 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676402 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676405 2558 flags.go:64] FLAG: --lock-file="" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676408 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676410 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676413 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676418 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676421 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676424 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676426 2558 flags.go:64] FLAG: --logging-format="text" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676429 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676432 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676434 2558 flags.go:64] FLAG: --manifest-url="" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676437 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676442 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676445 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676449 2558 flags.go:64] FLAG: --max-pods="110" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676452 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676454 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676457 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 20:46:54.682479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676460 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676463 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676467 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676470 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676478 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676481 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676484 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676487 2558 flags.go:64] FLAG: --pod-cidr="" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676489 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676495 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676498 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676501 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676504 2558 flags.go:64] FLAG: --port="10250" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676507 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676510 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0307d5de32c9a73f8" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676512 2558 flags.go:64] FLAG: --qos-reserved="" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676515 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676518 2558 flags.go:64] FLAG: --register-node="true" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676521 2558 flags.go:64] FLAG: --register-with-taints="" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676525 2558 flags.go:64] FLAG: --registry-burst="10" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676527 2558 flags.go:64] FLAG: --registry-qps="5" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676531 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676534 2558 flags.go:64] FLAG: --reserved-memory="" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676537 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676540 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 20:46:54.683107 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676543 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676546 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676549 2558 flags.go:64] FLAG: --runonce="false" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676551 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676554 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676557 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676560 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676563 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676566 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676570 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676573 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676576 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676579 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676582 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676584 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676587 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676590 2558 flags.go:64] FLAG: --system-cgroups="" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676593 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676598 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676601 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676603 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676608 2558 flags.go:64] FLAG: --tls-min-version="" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676626 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676631 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676636 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 20:46:54.683728 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676640 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676643 2558 flags.go:64] FLAG: --v="2" Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676648 2558 flags.go:64] FLAG: --version="false" Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676652 2558 flags.go:64] FLAG: --vmodule="" Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676657 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.676660 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676778 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676783 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676786 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676789 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676791 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676794 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676797 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676799 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676801 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676804 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676807 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676810 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676812 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676815 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 20:46:54.684378 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676817 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676819 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676822 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676825 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676827 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676830 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676832 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676835 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676838 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676842 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676845 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676848 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676851 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676854 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676856 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676859 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676861 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676865 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676867 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 20:46:54.684882 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676870 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676873 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676876 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676878 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676881 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676883 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676886 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676890 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676893 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676896 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676898 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676901 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676904 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676908 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676911 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676913 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676916 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676920 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676923 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 20:46:54.685339 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676926 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676929 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676931 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676934 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676936 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676939 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676941 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676944 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676946 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676949 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676952 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676954 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676957 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676959 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676962 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676965 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676967 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676969 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676972 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676974 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676977 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 20:46:54.685829 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676980 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676983 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676985 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676988 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676990 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676993 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676995 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.676997 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677000 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677002 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677004 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677007 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677009 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677012 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677014 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677016 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677019 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677021 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677024 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677026 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 20:46:54.686371 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677028 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677031 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677033 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677036 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677039 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.677041 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.677046 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.683781 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.683796 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.684646 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.684656 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684715 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684720 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684723 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 20:46:54.686877 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684727 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684732 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684735 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684738 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684741 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684745 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684747 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684750 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684752 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684755 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684758 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684761 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684763 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684766 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684768 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684772 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684777 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684781 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 20:46:54.687227 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684784 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684787 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684790 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684792 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684795 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684797 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684800 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684802 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684805 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684808 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684810 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684813 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684815 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684818 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684820 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684824 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684826 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684829 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684831 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684834 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 20:46:54.687689 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684837 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684840 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684842 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684844 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684847 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684849 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684852 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684854 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684856 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684859 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684861 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684863 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684866 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684868 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684871 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684873 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684876 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684878 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684881 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684884 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 20:46:54.688163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684886 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684890 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684892 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684895 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684897 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684900 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684902 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684905 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684908 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684910 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684912 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684915 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684917 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684919 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684922 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684924 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684926 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684929 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684931 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684933 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 20:46:54.688726 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684936 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684938 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684940 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684943 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684945 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684948 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684950 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684952 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684955 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684957 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684959 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684962 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684964 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684967 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684970 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684973 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684975 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 20:46:54.689216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.684977 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.684982 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685092 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685096 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685099 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685102 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685105 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685107 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685110 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685113 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685115 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685119 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685123 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685126 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685128 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 20:46:54.689678 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685131 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685133 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685136 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685138 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685141 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685143 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685146 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685148 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685150 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685153 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685155 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685158 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685160 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685162 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685165 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685168 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685170 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685172 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685175 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 20:46:54.690142 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685178 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685180 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685183 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685185 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685188 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685190 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685192 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685195 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685197 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685200 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685202 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685205 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685207 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685210 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685213 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685215 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685217 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685220 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685222 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685225 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 20:46:54.690738 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685227 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685229 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685232 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685234 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685237 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685239 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685241 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685244 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685246 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685249 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685251 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685254 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685256 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685259 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685261 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685264 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685266 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685269 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685271 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685273 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 20:46:54.691221 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685276 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685280 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685283 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685285 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685287 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685290 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685293 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685295 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685298 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685300 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685302 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685305 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685307 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685311 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685313 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685316 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685318 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685322 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685325 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685328 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 20:46:54.691807 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685330 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685333 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685337 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685341 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685344 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685347 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:54.685350 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.685355 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.685472 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.688137 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.689515 2558 server.go:1019] "Starting client certificate rotation" Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.689644 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 20:46:54.692400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.689686 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 20:46:54.692914 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.692705 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 20:46:54.693854 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.693836 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 20:46:54.704547 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.704530 2558 log.go:25] "Validated CRI v1 runtime API" Jul 01 20:46:54.708482 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.708466 2558 log.go:25] "Validated CRI v1 image API" Jul 01 20:46:54.709406 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.709379 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 20:46:54.713559 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.713542 2558 fs.go:135] Filesystem UUIDs: map[00a6f12a-6ba9-4465-b0bf-8c408c4210f1:/dev/nvme0n1p4 3a772c1d-6d2a-4628-a68e-a6ae4584e750:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 01 20:46:54.713645 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.713560 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 20:46:54.718969 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.718864 2558 manager.go:216] Machine: {Timestamp:2026-07-01 20:46:54.717066351 +0000 UTC m=+0.340152290 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095809 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21bb3796c082a3441f41c2ddf2fb99 SystemUUID:ec21bb37-96c0-82a3-441f-41c2ddf2fb99 BootID:240f0d10-c286-4bf7-a7d3-1490af5d2ec7 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7e:0f:cd:0e:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7e:0f:cd:0e:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:0f:73:ed:28:86 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 20:46:54.718969 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.718967 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 20:46:54.719087 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.719076 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 20:46:54.722767 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.722745 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 20:46:54.722903 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.722768 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-150.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 20:46:54.724152 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.724141 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 20:46:54.724184 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.724156 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 20:46:54.724184 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.724164 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 20:46:54.724760 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.724741 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 20:46:54.725601 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.725592 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 20:46:54.726143 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.726128 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 20:46:54.728496 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.728485 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 01 20:46:54.728609 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.728600 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 20:46:54.728657 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.728640 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 01 20:46:54.728657 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.728650 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 20:46:54.728717 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.728665 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 20:46:54.728717 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.728677 2558 kubelet.go:398] "Adding apiserver pod source" Jul 01 20:46:54.728717 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.728690 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 20:46:54.729047 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.729035 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 20:46:54.729082 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.729055 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 20:46:54.731260 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.731238 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 20:46:54.732648 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.732634 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 20:46:54.732714 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.732664 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 20:46:54.733260 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733220 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 20:46:54.733260 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733236 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 20:46:54.733260 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733250 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 20:46:54.733260 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733258 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 20:46:54.733389 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733279 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 20:46:54.733389 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733289 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 20:46:54.733389 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733352 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 20:46:54.733389 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733362 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 20:46:54.733389 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733374 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 20:46:54.733389 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733383 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 20:46:54.733533 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733404 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 20:46:54.733533 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.733416 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 20:46:54.734227 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.734215 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 20:46:54.734269 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.734230 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 20:46:54.737747 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.737735 2558 server.go:1268] "Started kubelet" Jul 01 20:46:54.737867 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.737804 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 20:46:54.737925 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.737890 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 20:46:54.738077 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.737848 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 20:46:54.738560 ip-10-0-130-150 systemd[1]: Started Kubernetes Kubelet. Jul 01 20:46:54.739443 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.739054 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 20:46:54.739443 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.739271 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 01 20:46:54.743428 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.743398 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 20:46:54.743428 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.743406 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-150.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 20:46:54.743986 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.743956 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-150.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 20:46:54.744076 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.743956 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 20:46:54.746378 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.746358 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 20:46:54.746378 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.746366 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 20:46:54.746517 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.746464 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:54.747065 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747048 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 20:46:54.747158 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747068 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 20:46:54.747158 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747052 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 20:46:54.747158 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747142 2558 factory.go:55] Registering systemd factory Jul 01 20:46:54.747158 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747155 2558 factory.go:223] Registration of the systemd container factory successfully Jul 01 20:46:54.747351 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747167 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 01 20:46:54.747351 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747174 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 01 20:46:54.747434 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747357 2558 factory.go:153] Registering CRI-O factory Jul 01 20:46:54.747434 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747373 2558 factory.go:223] Registration of the crio container factory successfully Jul 01 20:46:54.747434 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747417 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 20:46:54.747536 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747439 2558 factory.go:103] Registering Raw factory Jul 01 20:46:54.747536 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747453 2558 manager.go:1196] Started watching for new ooms in manager Jul 01 20:46:54.747864 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.747850 2558 manager.go:349] Starting recovery of all containers Jul 01 20:46:54.755037 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.754993 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-150.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 20:46:54.756040 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.756011 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 20:46:54.757204 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.755937 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-150.ec2.internal.18be4567d4b27679 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-150.ec2.internal,UID:ip-10-0-130-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-150.ec2.internal,},FirstTimestamp:2026-07-01 20:46:54.737708665 +0000 UTC m=+0.360794552,LastTimestamp:2026-07-01 20:46:54.737708665 +0000 UTC m=+0.360794552,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-150.ec2.internal,}" Jul 01 20:46:54.758136 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.758119 2558 manager.go:354] Recovery completed Jul 01 20:46:54.762908 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.762895 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 20:46:54.764635 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.764604 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientMemory" Jul 01 20:46:54.764703 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.764651 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 20:46:54.764703 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.764664 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientPID" Jul 01 20:46:54.765343 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.765331 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 20:46:54.765343 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.765343 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 20:46:54.765430 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.765360 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 01 20:46:54.765430 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.765343 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 20:46:54.766164 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.766149 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientMemory" Jul 01 20:46:54.766239 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.766175 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 20:46:54.766239 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.766185 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientPID" Jul 01 20:46:54.766557 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.766546 2558 policy_none.go:49] "None policy: Start" Jul 01 20:46:54.766593 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.766561 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 20:46:54.766593 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.766571 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 20:46:54.767259 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.767154 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-150.ec2.internal.18be4567d64d53d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-150.ec2.internal,UID:ip-10-0-130-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-150.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-150.ec2.internal,},FirstTimestamp:2026-07-01 20:46:54.76463509 +0000 UTC m=+0.387720979,LastTimestamp:2026-07-01 20:46:54.76463509 +0000 UTC m=+0.387720979,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-150.ec2.internal,}" Jul 01 20:46:54.767698 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.767687 2558 policy_none.go:47] "Start" Jul 01 20:46:54.773097 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.773033 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-150.ec2.internal.18be4567d64db0cb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-150.ec2.internal,UID:ip-10-0-130-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-150.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-150.ec2.internal,},FirstTimestamp:2026-07-01 20:46:54.764658891 +0000 UTC m=+0.387744779,LastTimestamp:2026-07-01 20:46:54.764658891 +0000 UTC m=+0.387744779,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-150.ec2.internal,}" Jul 01 20:46:54.778179 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.778114 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-150.ec2.internal.18be4567d64dd688 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-150.ec2.internal,UID:ip-10-0-130-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-150.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-150.ec2.internal,},FirstTimestamp:2026-07-01 20:46:54.764668552 +0000 UTC m=+0.387754440,LastTimestamp:2026-07-01 20:46:54.764668552 +0000 UTC m=+0.387754440,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-150.ec2.internal,}" Jul 01 20:46:54.781331 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.781272 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-150.ec2.internal.18be4567d64d53d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-150.ec2.internal.18be4567d64d53d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-150.ec2.internal,UID:ip-10-0-130-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-150.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-150.ec2.internal,},FirstTimestamp:2026-07-01 20:46:54.76463509 +0000 UTC m=+0.387720979,LastTimestamp:2026-07-01 20:46:54.76616175 +0000 UTC m=+0.389247637,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-150.ec2.internal,}" Jul 01 20:46:54.791460 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.791396 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-150.ec2.internal.18be4567d64db0cb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-150.ec2.internal.18be4567d64db0cb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-150.ec2.internal,UID:ip-10-0-130-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-150.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-150.ec2.internal,},FirstTimestamp:2026-07-01 20:46:54.764658891 +0000 UTC m=+0.387744779,LastTimestamp:2026-07-01 20:46:54.76617968 +0000 UTC m=+0.389265568,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-150.ec2.internal,}" Jul 01 20:46:54.805238 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.805222 2558 manager.go:337] "Starting Device Plugin manager" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.805265 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.805279 2558 server.go:87] "Starting device plugin registration server" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.805489 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.805497 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.805604 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.805728 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.805738 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.806193 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.806413 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.807785 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-150.ec2.internal.18be4567d64dd688\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-150.ec2.internal.18be4567d64dd688 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-150.ec2.internal,UID:ip-10-0-130-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-150.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-150.ec2.internal,},FirstTimestamp:2026-07-01 20:46:54.764668552 +0000 UTC m=+0.387754440,LastTimestamp:2026-07-01 20:46:54.76618887 +0000 UTC m=+0.389274758,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-150.ec2.internal,}" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.818696 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-150.ec2.internal.18be4567d8ddc8eb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-150.ec2.internal,UID:ip-10-0-130-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-130-150.ec2.internal,},FirstTimestamp:2026-07-01 20:46:54.807656683 +0000 UTC m=+0.430742558,LastTimestamp:2026-07-01 20:46:54.807656683 +0000 UTC m=+0.430742558,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-150.ec2.internal,}" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.821952 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.823024 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g4xzl" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.823153 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.823182 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.823199 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.823227 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.833786 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 01 20:46:54.843328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.836302 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g4xzl" Jul 01 20:46:54.906370 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.906342 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 20:46:54.907584 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.907565 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientMemory" Jul 01 20:46:54.907690 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.907598 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 20:46:54.907690 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.907609 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientPID" Jul 01 20:46:54.907690 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.907657 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.923594 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.923543 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal"] Jul 01 20:46:54.923679 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.923600 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 20:46:54.925016 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.925003 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientMemory" Jul 01 20:46:54.925073 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.925034 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 20:46:54.925073 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.925046 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientPID" Jul 01 20:46:54.926346 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.926335 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 20:46:54.926454 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.926439 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.926506 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.926467 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 20:46:54.927067 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.927051 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientMemory" Jul 01 20:46:54.927067 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.927064 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientMemory" Jul 01 20:46:54.927201 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.927083 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 20:46:54.927201 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.927088 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 20:46:54.927201 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.927097 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientPID" Jul 01 20:46:54.927201 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.927098 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientPID" Jul 01 20:46:54.928095 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.928082 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.928140 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.928100 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-150.ec2.internal\": node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:54.928341 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.928328 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.928388 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.928356 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 20:46:54.929100 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.929082 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientMemory" Jul 01 20:46:54.929192 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.929115 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 20:46:54.929192 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.929131 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientPID" Jul 01 20:46:54.940220 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.940194 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-150.ec2.internal\" not found" node="ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.943905 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.943888 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-150.ec2.internal\" not found" node="ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.945697 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:54.945681 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:54.948530 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.948516 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0316c5dcf0b57b896d9dfde967cd3a4e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal\" (UID: \"0316c5dcf0b57b896d9dfde967cd3a4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.948595 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.948538 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0316c5dcf0b57b896d9dfde967cd3a4e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal\" (UID: \"0316c5dcf0b57b896d9dfde967cd3a4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:54.948595 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:54.948554 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c3f741d9ca74dc13685dec19f2b0ff2d-config\") pod \"kube-apiserver-proxy-ip-10-0-130-150.ec2.internal\" (UID: \"c3f741d9ca74dc13685dec19f2b0ff2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.061426 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.061406 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.063635 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.063622 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0316c5dcf0b57b896d9dfde967cd3a4e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal\" (UID: \"0316c5dcf0b57b896d9dfde967cd3a4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.063697 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.063643 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0316c5dcf0b57b896d9dfde967cd3a4e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal\" (UID: \"0316c5dcf0b57b896d9dfde967cd3a4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.063697 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.063660 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c3f741d9ca74dc13685dec19f2b0ff2d-config\") pod \"kube-apiserver-proxy-ip-10-0-130-150.ec2.internal\" (UID: \"c3f741d9ca74dc13685dec19f2b0ff2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.063795 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.063703 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c3f741d9ca74dc13685dec19f2b0ff2d-config\") pod \"kube-apiserver-proxy-ip-10-0-130-150.ec2.internal\" (UID: \"c3f741d9ca74dc13685dec19f2b0ff2d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.063795 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.063737 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0316c5dcf0b57b896d9dfde967cd3a4e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal\" (UID: \"0316c5dcf0b57b896d9dfde967cd3a4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.063795 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.063743 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0316c5dcf0b57b896d9dfde967cd3a4e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal\" (UID: \"0316c5dcf0b57b896d9dfde967cd3a4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.162217 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.162147 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.257910 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.257833 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.262320 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.262028 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" Jul 01 20:46:56.263211 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.263191 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.364128 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.364092 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.464691 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.464658 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.565129 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.565050 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.665646 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.665620 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.703930 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.703907 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 20:46:56.766467 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.766392 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.771562 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.771539 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 20:46:56.785061 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:56.785034 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f741d9ca74dc13685dec19f2b0ff2d.slice/crio-44131cf1216f19e6717530b6d665020b9a8dbc2f41bd884d3d692f7a19ee7d93 WatchSource:0}: Error finding container 44131cf1216f19e6717530b6d665020b9a8dbc2f41bd884d3d692f7a19ee7d93: Status 404 returned error can't find the container with id 44131cf1216f19e6717530b6d665020b9a8dbc2f41bd884d3d692f7a19ee7d93 Jul 01 20:46:56.786639 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:56.786611 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0316c5dcf0b57b896d9dfde967cd3a4e.slice/crio-990e32a0a8062a65762df852884d7797cf1204c4fa9297ec525e908aa25faa1f WatchSource:0}: Error finding container 990e32a0a8062a65762df852884d7797cf1204c4fa9297ec525e908aa25faa1f: Status 404 returned error can't find the container with id 990e32a0a8062a65762df852884d7797cf1204c4fa9297ec525e908aa25faa1f Jul 01 20:46:56.790454 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.790429 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 20:46:56.840815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.840732 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" event={"ID":"0316c5dcf0b57b896d9dfde967cd3a4e","Type":"ContainerStarted","Data":"990e32a0a8062a65762df852884d7797cf1204c4fa9297ec525e908aa25faa1f"} Jul 01 20:46:56.841700 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.841682 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" event={"ID":"c3f741d9ca74dc13685dec19f2b0ff2d","Type":"ContainerStarted","Data":"44131cf1216f19e6717530b6d665020b9a8dbc2f41bd884d3d692f7a19ee7d93"} Jul 01 20:46:56.858793 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.858770 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 20:46:56.860721 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.860699 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 20:41:55 +0000 UTC" deadline="2028-03-27 23:14:18.55871668 +0000 UTC" Jul 01 20:46:56.860780 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.860722 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15242h27m21.697997483s" Jul 01 20:46:56.861003 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.860992 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientMemory" Jul 01 20:46:56.861040 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.861017 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 20:46:56.861040 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.861027 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeHasSufficientPID" Jul 01 20:46:56.866603 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.866590 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.966944 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:56.966915 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:56.993218 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:56.993197 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 20:46:57.067895 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.067871 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:57.084788 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.084768 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 20:46:57.168984 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.168913 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-150.ec2.internal\" not found" Jul 01 20:46:57.227726 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.227699 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 20:46:57.262037 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.261990 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" Jul 01 20:46:57.277171 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.277144 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 20:46:57.278008 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.277986 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" Jul 01 20:46:57.291633 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.291589 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 20:46:57.744792 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.744755 2558 apiserver.go:52] "Watching apiserver" Jul 01 20:46:57.751596 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.751571 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 20:46:57.753751 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.753726 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-zmn82","kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz","openshift-cluster-node-tuning-operator/tuned-4k29z","openshift-dns/node-resolver-8k6wq","openshift-network-diagnostics/network-check-target-xj5kn","openshift-ovn-kubernetes/ovnkube-node-6r267","openshift-image-registry/node-ca-m5fg2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal","openshift-multus/multus-64tzn","openshift-multus/multus-additional-cni-plugins-8wkrd","openshift-multus/network-metrics-daemon-kkfpv","openshift-network-operator/iptables-alerter-9k6kh","kube-system/global-pull-secret-syncer-2ntmn"] Jul 01 20:46:57.755486 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.755461 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.756702 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.756677 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:57.756802 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.756754 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:46:57.757962 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.757945 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 20:46:57.758418 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.758399 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4stpj\"" Jul 01 20:46:57.758718 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.758699 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 20:46:57.758839 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.758824 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 20:46:57.759124 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.759106 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.760574 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.760550 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.761066 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.761049 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 20:46:57.761392 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.761363 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 20:46:57.761571 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.761552 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 20:46:57.761842 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.761713 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 20:46:57.761842 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.761756 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c5knz\"" Jul 01 20:46:57.761842 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.761765 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 20:46:57.761842 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.761797 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 20:46:57.762061 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.761971 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.762634 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.762604 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ts7ck\"" Jul 01 20:46:57.764864 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.764847 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 20:46:57.764991 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.764974 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 20:46:57.765193 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.765177 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:46:57.765248 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.765229 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:46:57.765690 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.765673 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 20:46:57.767627 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.767582 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zpvvw\"" Jul 01 20:46:57.768011 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.767984 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.769352 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.769337 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.769972 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.769957 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 20:46:57.770478 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.770457 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 20:46:57.770626 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.770610 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 20:46:57.770766 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.770751 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 20:46:57.771213 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.771198 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:46:57.771331 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.771317 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 20:46:57.771395 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.771326 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5drx9\"" Jul 01 20:46:57.771664 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.771649 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 20:46:57.773295 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.772876 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 20:46:57.773295 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.773128 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 20:46:57.773593 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.773560 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.774325 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.773703 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mjhtd\"" Jul 01 20:46:57.774325 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.773782 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-svsdd\"" Jul 01 20:46:57.774325 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.774155 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 20:46:57.774325 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.774239 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775645 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-system-cni-dir\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775676 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775707 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775738 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-cni-binary-copy\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775789 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-socket-dir-parent\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775838 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-netns\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775864 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775890 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-daemon-config\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775909 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775928 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-os-release\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775949 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.775963 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-cni-multus\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776026 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-tmp\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776047 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h4xlf\"" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776057 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hrb9\" (UniqueName: \"kubernetes.io/projected/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-kube-api-access-2hrb9\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776092 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5b40ef0-7439-42ae-a63e-30d2c415238b-host\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.776186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776107 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e5b40ef0-7439-42ae-a63e-30d2c415238b-serviceca\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776125 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-hostroot\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-systemd\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776365 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776599 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776736 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-sys\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776757 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdn6m\" (UniqueName: \"kubernetes.io/projected/220e4077-2a30-4706-863a-6ec30801b3f3-kube-api-access-rdn6m\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776775 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-cni-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776820 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-cnibin\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776855 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-kubelet\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776883 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-modprobe-d\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.777093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.776944 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-var-lib-kubelet\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.778765 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.778418 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktdp4\" (UniqueName: \"kubernetes.io/projected/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-kube-api-access-ktdp4\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.778765 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.778539 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.778765 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.778679 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-cnibin\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.779439 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.779347 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-os-release\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.779520 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.779490 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysctl-d\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.779746 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.779726 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysctl-conf\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.780767 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-56hw5\"" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781724 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-tuned\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781811 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781792 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-multus-certs\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781438 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781886 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6tsq\" (UniqueName: \"kubernetes.io/projected/2212862b-69c4-4886-a071-54cf439b9fd0-kube-api-access-d6tsq\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781510 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781934 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-cni-bin\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781231 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781967 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-conf-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.781997 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysconfig\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.781996 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782037 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-kubernetes\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782063 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-host\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782084 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-system-cni-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782109 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-etc-kubernetes\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782134 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-run\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782175 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9dpq\" (UniqueName: \"kubernetes.io/projected/e5b40ef0-7439-42ae-a63e-30d2c415238b-kube-api-access-r9dpq\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.782368 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782213 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-k8s-cni-cncf-io\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.782985 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-lib-modules\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.782985 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782295 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:57.782985 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.782308 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 20:46:57.824169 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.824146 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hfz8j" Jul 01 20:46:57.835320 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.835301 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hfz8j" Jul 01 20:46:57.862678 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.862651 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 20:46:57.882624 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882585 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9dpq\" (UniqueName: \"kubernetes.io/projected/e5b40ef0-7439-42ae-a63e-30d2c415238b-kube-api-access-r9dpq\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882632 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-etc-selinux\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882658 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-multus-certs\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6tsq\" (UniqueName: \"kubernetes.io/projected/2212862b-69c4-4886-a071-54cf439b9fd0-kube-api-access-d6tsq\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-kubernetes\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882731 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-host\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882732 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-multus-certs\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882752 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-sys-fs\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882776 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1172c283-88c3-407e-8c99-4af696b5984c-dbus\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.882814 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-etc-kubernetes\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882822 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-run\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882828 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-host\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882844 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-registration-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882876 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-kubernetes\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882883 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-cni-bin\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882901 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-etc-kubernetes\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882938 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4-konnectivity-ca\") pod \"konnectivity-agent-zmn82\" (UID: \"4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4\") " pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.882964 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-lib-modules\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883032 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-run\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883062 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883092 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883121 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovnkube-script-lib\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883063 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-lib-modules\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.883136 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883149 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883174 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.883205 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.883205 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs podName:220e4077-2a30-4706-863a-6ec30801b3f3 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:58.383180205 +0000 UTC m=+2.991623059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs") pod "network-metrics-daemon-kkfpv" (UID: "220e4077-2a30-4706-863a-6ec30801b3f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883233 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-cni-binary-copy\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883280 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-netns\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883303 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883325 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e5b40ef0-7439-42ae-a63e-30d2c415238b-serviceca\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jzb2\" (UniqueName: \"kubernetes.io/projected/6c605213-6328-49f7-b4e0-3a5283e3d27b-kube-api-access-7jzb2\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883362 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-netns\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883375 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1172c283-88c3-407e-8c99-4af696b5984c-kubelet-config\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883380 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883400 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883434 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-cni-multus\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883456 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hrb9\" (UniqueName: \"kubernetes.io/projected/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-kube-api-access-2hrb9\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883475 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdn6m\" (UniqueName: \"kubernetes.io/projected/220e4077-2a30-4706-863a-6ec30801b3f3-kube-api-access-rdn6m\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883512 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-cni-multus\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883531 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6a61d92-4e50-4935-9291-4a42de8fb80e-host-slash\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883551 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-run-ovn-kubernetes\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883681 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsxk7\" (UniqueName: \"kubernetes.io/projected/46646e21-8a7b-4443-98c5-5e5556d69ceb-kube-api-access-gsxk7\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.883834 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883758 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4-agent-certs\") pod \"konnectivity-agent-zmn82\" (UID: \"4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4\") " pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883763 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883806 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-cni-binary-copy\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883860 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e5b40ef0-7439-42ae-a63e-30d2c415238b-serviceca\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883871 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-systemd\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883997 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-systemd\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.883995 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884038 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-socket-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884068 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884104 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-cni-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884120 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884130 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-var-lib-kubelet\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884158 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-slash\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884182 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884191 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-var-lib-kubelet\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884233 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-ovn\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.884535 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884281 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-cni-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884286 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2212862b-69c4-4886-a071-54cf439b9fd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884300 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-cni-netd\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884345 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-cnibin\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884370 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysctl-d\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884377 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-cnibin\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884394 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysctl-conf\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884417 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-tuned\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884442 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-etc-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884476 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysctl-d\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884466 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6c605213-6328-49f7-b4e0-3a5283e3d27b-hosts-file\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884533 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysctl-conf\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884535 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-device-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884569 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-cni-bin\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884584 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-conf-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884598 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysconfig\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884620 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-cni-bin\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885231 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884638 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-conf-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884621 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-system-cni-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884660 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-system-cni-dir\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884651 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-sysconfig\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884675 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-log-socket\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884700 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovnkube-config\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884721 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-k8s-cni-cncf-io\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884737 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bc46\" (UniqueName: \"kubernetes.io/projected/c6a61d92-4e50-4935-9291-4a42de8fb80e-kube-api-access-2bc46\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884744 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-run-k8s-cni-cncf-io\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884765 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-systemd\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884785 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-system-cni-dir\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884766 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884800 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-socket-dir-parent\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884814 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5b40ef0-7439-42ae-a63e-30d2c415238b-host\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884828 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-daemon-config\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884842 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-os-release\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884865 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-tmp\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884867 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-socket-dir-parent\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.885815 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884871 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5b40ef0-7439-42ae-a63e-30d2c415238b-host\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884841 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-system-cni-dir\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884891 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-systemd-units\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884947 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovn-node-metrics-cert\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884904 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-os-release\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884964 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-hostroot\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.884993 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-sys\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885011 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-hostroot\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885014 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhdkb\" (UniqueName: \"kubernetes.io/projected/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-kube-api-access-dhdkb\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885049 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-kubelet\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885072 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-sys\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885074 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-cnibin\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885120 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-cnibin\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885131 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-kubelet\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885165 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-modprobe-d\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885185 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-host-var-lib-kubelet\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885193 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-run-netns\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885220 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-var-lib-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.886500 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885240 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-node-log\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.887031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885245 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-multus-daemon-config\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.887031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885276 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-modprobe-d\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.887031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885280 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-env-overrides\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.887031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885303 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktdp4\" (UniqueName: \"kubernetes.io/projected/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-kube-api-access-ktdp4\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.887031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885326 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-os-release\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.887031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885379 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c605213-6328-49f7-b4e0-3a5283e3d27b-tmp-dir\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.887031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885396 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c6a61d92-4e50-4935-9291-4a42de8fb80e-iptables-alerter-script\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.887031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.885396 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2212862b-69c4-4886-a071-54cf439b9fd0-os-release\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.887991 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.887972 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-etc-tuned\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.888100 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.888085 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-tmp\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.890269 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.890238 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 20:46:57.890269 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.890273 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 20:46:57.890401 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.890285 2558 projected.go:196] Error preparing data for projected volume kube-api-access-846pb for pod openshift-network-diagnostics/network-check-target-xj5kn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:46:57.890401 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.890348 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb podName:af091fbd-26ff-4648-9042-fa91ba493331 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:58.390331933 +0000 UTC m=+2.998774787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-846pb" (UniqueName: "kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb") pod "network-check-target-xj5kn" (UID: "af091fbd-26ff-4648-9042-fa91ba493331") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:46:57.892763 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.892702 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9dpq\" (UniqueName: \"kubernetes.io/projected/e5b40ef0-7439-42ae-a63e-30d2c415238b-kube-api-access-r9dpq\") pod \"node-ca-m5fg2\" (UID: \"e5b40ef0-7439-42ae-a63e-30d2c415238b\") " pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:57.893377 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.893352 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6tsq\" (UniqueName: \"kubernetes.io/projected/2212862b-69c4-4886-a071-54cf439b9fd0-kube-api-access-d6tsq\") pod \"multus-additional-cni-plugins-8wkrd\" (UID: \"2212862b-69c4-4886-a071-54cf439b9fd0\") " pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:57.894174 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.894152 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hrb9\" (UniqueName: \"kubernetes.io/projected/98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa-kube-api-access-2hrb9\") pod \"tuned-4k29z\" (UID: \"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa\") " pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:57.894727 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.894703 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktdp4\" (UniqueName: \"kubernetes.io/projected/cf1d7d59-659b-43a0-ad53-cee5333fd5fe-kube-api-access-ktdp4\") pod \"multus-64tzn\" (UID: \"cf1d7d59-659b-43a0-ad53-cee5333fd5fe\") " pod="openshift-multus/multus-64tzn" Jul 01 20:46:57.895474 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.895452 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdn6m\" (UniqueName: \"kubernetes.io/projected/220e4077-2a30-4706-863a-6ec30801b3f3-kube-api-access-rdn6m\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:57.986313 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986279 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-ovn\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986313 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986309 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-cni-netd\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986328 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-etc-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986342 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6c605213-6328-49f7-b4e0-3a5283e3d27b-hosts-file\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986358 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-device-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986380 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-log-socket\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986400 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovnkube-config\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986403 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-ovn\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986425 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bc46\" (UniqueName: \"kubernetes.io/projected/c6a61d92-4e50-4935-9291-4a42de8fb80e-kube-api-access-2bc46\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986440 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6c605213-6328-49f7-b4e0-3a5283e3d27b-hosts-file\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986396 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-cni-netd\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986458 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-systemd\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986405 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-etc-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986444 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-log-socket\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986443 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-device-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986496 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-systemd\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986533 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-systemd-units\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.986554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986557 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovn-node-metrics-cert\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986583 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhdkb\" (UniqueName: \"kubernetes.io/projected/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-kube-api-access-dhdkb\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986605 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-kubelet\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986628 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-run-netns\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986650 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-var-lib-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986670 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-node-log\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986693 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-env-overrides\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986716 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c605213-6328-49f7-b4e0-3a5283e3d27b-tmp-dir\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c6a61d92-4e50-4935-9291-4a42de8fb80e-iptables-alerter-script\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986766 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-etc-selinux\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986771 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-var-lib-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986792 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-sys-fs\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986816 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1172c283-88c3-407e-8c99-4af696b5984c-dbus\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986840 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-registration-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986862 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-cni-bin\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986883 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4-konnectivity-ca\") pod \"konnectivity-agent-zmn82\" (UID: \"4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4\") " pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986923 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.987440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986948 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovnkube-config\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986965 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-kubelet\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986952 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovnkube-script-lib\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.986974 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-etc-selinux\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987025 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-cni-bin\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987024 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jzb2\" (UniqueName: \"kubernetes.io/projected/6c605213-6328-49f7-b4e0-3a5283e3d27b-kube-api-access-7jzb2\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987061 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1172c283-88c3-407e-8c99-4af696b5984c-kubelet-config\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987095 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6a61d92-4e50-4935-9291-4a42de8fb80e-host-slash\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-run-ovn-kubernetes\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987142 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsxk7\" (UniqueName: \"kubernetes.io/projected/46646e21-8a7b-4443-98c5-5e5556d69ceb-kube-api-access-gsxk7\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987163 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4-agent-certs\") pod \"konnectivity-agent-zmn82\" (UID: \"4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4\") " pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987189 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-socket-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987242 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987286 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-slash\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987310 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987376 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-run-openvswitch\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.988285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987411 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-env-overrides\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987481 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovnkube-script-lib\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987661 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-systemd-units\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987689 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c6a61d92-4e50-4935-9291-4a42de8fb80e-iptables-alerter-script\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987713 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-run-netns\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987765 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-sys-fs\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987808 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-registration-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987926 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987966 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1172c283-88c3-407e-8c99-4af696b5984c-kubelet-config\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.987990 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6a61d92-4e50-4935-9291-4a42de8fb80e-host-slash\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.988012 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-run-ovn-kubernetes\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.988338 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4-konnectivity-ca\") pod \"konnectivity-agent-zmn82\" (UID: \"4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4\") " pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.988451 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.988543 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-socket-dir\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.988595 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.988651 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-node-log\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.988774 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46646e21-8a7b-4443-98c5-5e5556d69ceb-host-slash\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.989109 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:57.988811 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret podName:1172c283-88c3-407e-8c99-4af696b5984c nodeName:}" failed. No retries permitted until 2026-07-01 20:46:58.488791259 +0000 UTC m=+3.097234127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret") pod "global-pull-secret-syncer-2ntmn" (UID: "1172c283-88c3-407e-8c99-4af696b5984c") : object "kube-system"/"original-pull-secret" not registered Jul 01 20:46:57.989976 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.988839 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1172c283-88c3-407e-8c99-4af696b5984c-dbus\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:57.989976 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.988930 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c605213-6328-49f7-b4e0-3a5283e3d27b-tmp-dir\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:57.991540 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.991517 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46646e21-8a7b-4443-98c5-5e5556d69ceb-ovn-node-metrics-cert\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:57.991624 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.991588 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4-agent-certs\") pod \"konnectivity-agent-zmn82\" (UID: \"4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4\") " pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:46:57.995774 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:57.995708 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bc46\" (UniqueName: \"kubernetes.io/projected/c6a61d92-4e50-4935-9291-4a42de8fb80e-kube-api-access-2bc46\") pod \"iptables-alerter-9k6kh\" (UID: \"c6a61d92-4e50-4935-9291-4a42de8fb80e\") " pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:58.001938 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.001900 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsxk7\" (UniqueName: \"kubernetes.io/projected/46646e21-8a7b-4443-98c5-5e5556d69ceb-kube-api-access-gsxk7\") pod \"ovnkube-node-6r267\" (UID: \"46646e21-8a7b-4443-98c5-5e5556d69ceb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:58.002052 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.001939 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhdkb\" (UniqueName: \"kubernetes.io/projected/a06cf41c-de16-46c4-a552-ffb04f2bdbfe-kube-api-access-dhdkb\") pod \"aws-ebs-csi-driver-node-9rjlz\" (UID: \"a06cf41c-de16-46c4-a552-ffb04f2bdbfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:58.003309 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.003290 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jzb2\" (UniqueName: \"kubernetes.io/projected/6c605213-6328-49f7-b4e0-3a5283e3d27b-kube-api-access-7jzb2\") pod \"node-resolver-8k6wq\" (UID: \"6c605213-6328-49f7-b4e0-3a5283e3d27b\") " pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:58.068447 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.068414 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m5fg2" Jul 01 20:46:58.076046 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.076018 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" Jul 01 20:46:58.085902 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.085861 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-64tzn" Jul 01 20:46:58.092670 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.092653 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4k29z" Jul 01 20:46:58.098701 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.098680 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:46:58.104239 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.104221 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8k6wq" Jul 01 20:46:58.110857 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.110840 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:46:58.117731 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.117715 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" Jul 01 20:46:58.123511 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.123486 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9k6kh" Jul 01 20:46:58.265328 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.265295 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ed8b15_b9a4_45e6_9b67_ad39e7abc7aa.slice/crio-86a1e950ea75d69596548198cb218397cbfe95b49815f335790f206a17b0d300 WatchSource:0}: Error finding container 86a1e950ea75d69596548198cb218397cbfe95b49815f335790f206a17b0d300: Status 404 returned error can't find the container with id 86a1e950ea75d69596548198cb218397cbfe95b49815f335790f206a17b0d300 Jul 01 20:46:58.267212 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.267071 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1d7d59_659b_43a0_ad53_cee5333fd5fe.slice/crio-ec032b9430afe933706efd7a039bc9db126222919e39355975b3f02efc83ee33 WatchSource:0}: Error finding container ec032b9430afe933706efd7a039bc9db126222919e39355975b3f02efc83ee33: Status 404 returned error can't find the container with id ec032b9430afe933706efd7a039bc9db126222919e39355975b3f02efc83ee33 Jul 01 20:46:58.271391 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.270242 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46646e21_8a7b_4443_98c5_5e5556d69ceb.slice/crio-5c06363fbf9b2a10754a5ef2bb8591b9bcd63ec1fb6c6ab6efcb4ff9adcdcf28 WatchSource:0}: Error finding container 5c06363fbf9b2a10754a5ef2bb8591b9bcd63ec1fb6c6ab6efcb4ff9adcdcf28: Status 404 returned error can't find the container with id 5c06363fbf9b2a10754a5ef2bb8591b9bcd63ec1fb6c6ab6efcb4ff9adcdcf28 Jul 01 20:46:58.273140 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.273120 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a61d92_4e50_4935_9291_4a42de8fb80e.slice/crio-8627bbc44edf2bc2223bdf25bbc22e82d0c6e383e41895cacc9134287ace4578 WatchSource:0}: Error finding container 8627bbc44edf2bc2223bdf25bbc22e82d0c6e383e41895cacc9134287ace4578: Status 404 returned error can't find the container with id 8627bbc44edf2bc2223bdf25bbc22e82d0c6e383e41895cacc9134287ace4578 Jul 01 20:46:58.274216 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.273977 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2212862b_69c4_4886_a071_54cf439b9fd0.slice/crio-2182c429bca7c7dfbdd3e0d0fbbfc63880e335276ffa75c556b94e174acc0745 WatchSource:0}: Error finding container 2182c429bca7c7dfbdd3e0d0fbbfc63880e335276ffa75c556b94e174acc0745: Status 404 returned error can't find the container with id 2182c429bca7c7dfbdd3e0d0fbbfc63880e335276ffa75c556b94e174acc0745 Jul 01 20:46:58.274579 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.274558 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b40ef0_7439_42ae_a63e_30d2c415238b.slice/crio-a08f417bbca59c03d18db3bd18b0ddef79ebbe68adf569a170a8905ab2edd86d WatchSource:0}: Error finding container a08f417bbca59c03d18db3bd18b0ddef79ebbe68adf569a170a8905ab2edd86d: Status 404 returned error can't find the container with id a08f417bbca59c03d18db3bd18b0ddef79ebbe68adf569a170a8905ab2edd86d Jul 01 20:46:58.276059 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.275947 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c605213_6328_49f7_b4e0_3a5283e3d27b.slice/crio-d319653e4aed6397efd5ea06b55df858802659e3b86e77c62417580f63af3173 WatchSource:0}: Error finding container d319653e4aed6397efd5ea06b55df858802659e3b86e77c62417580f63af3173: Status 404 returned error can't find the container with id d319653e4aed6397efd5ea06b55df858802659e3b86e77c62417580f63af3173 Jul 01 20:46:58.277137 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.277102 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06cf41c_de16_46c4_a552_ffb04f2bdbfe.slice/crio-dbd97d7d3d4eb1d22084562da02d6f61bfd2b886e6cfb211cc318a65ebe91051 WatchSource:0}: Error finding container dbd97d7d3d4eb1d22084562da02d6f61bfd2b886e6cfb211cc318a65ebe91051: Status 404 returned error can't find the container with id dbd97d7d3d4eb1d22084562da02d6f61bfd2b886e6cfb211cc318a65ebe91051 Jul 01 20:46:58.279508 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:46:58.278389 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6d0ff1_f7c9_4bda_a541_3eb3abf901f4.slice/crio-18c21ea6ad47157b044c01902ecefef650f935de83bf32ce375d1cba8aafea73 WatchSource:0}: Error finding container 18c21ea6ad47157b044c01902ecefef650f935de83bf32ce375d1cba8aafea73: Status 404 returned error can't find the container with id 18c21ea6ad47157b044c01902ecefef650f935de83bf32ce375d1cba8aafea73 Jul 01 20:46:58.389532 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.389501 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:58.389707 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:58.389683 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:46:58.389810 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:58.389752 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs podName:220e4077-2a30-4706-863a-6ec30801b3f3 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:59.38973327 +0000 UTC m=+3.998176148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs") pod "network-metrics-daemon-kkfpv" (UID: "220e4077-2a30-4706-863a-6ec30801b3f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:46:58.490865 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.490826 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:46:58.491003 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.490886 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:58.491003 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:58.490987 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 20:46:58.491003 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:58.490991 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 20:46:58.491159 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:58.491020 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 20:46:58.491159 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:58.491036 2558 projected.go:196] Error preparing data for projected volume kube-api-access-846pb for pod openshift-network-diagnostics/network-check-target-xj5kn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:46:58.491159 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:58.491058 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret podName:1172c283-88c3-407e-8c99-4af696b5984c nodeName:}" failed. No retries permitted until 2026-07-01 20:46:59.491040015 +0000 UTC m=+4.099482872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret") pod "global-pull-secret-syncer-2ntmn" (UID: "1172c283-88c3-407e-8c99-4af696b5984c") : object "kube-system"/"original-pull-secret" not registered Jul 01 20:46:58.491159 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:58.491085 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb podName:af091fbd-26ff-4648-9042-fa91ba493331 nodeName:}" failed. No retries permitted until 2026-07-01 20:46:59.491069552 +0000 UTC m=+4.099512421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-846pb" (UniqueName: "kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb") pod "network-check-target-xj5kn" (UID: "af091fbd-26ff-4648-9042-fa91ba493331") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:46:58.839609 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.838385 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 20:41:57 +0000 UTC" deadline="2028-01-31 05:53:57.993156605 +0000 UTC" Jul 01 20:46:58.839609 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.839538 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13881h6m59.153629285s" Jul 01 20:46:58.849162 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.849125 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zmn82" event={"ID":"4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4","Type":"ContainerStarted","Data":"18c21ea6ad47157b044c01902ecefef650f935de83bf32ce375d1cba8aafea73"} Jul 01 20:46:58.852194 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.852087 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m5fg2" event={"ID":"e5b40ef0-7439-42ae-a63e-30d2c415238b","Type":"ContainerStarted","Data":"a08f417bbca59c03d18db3bd18b0ddef79ebbe68adf569a170a8905ab2edd86d"} Jul 01 20:46:58.854152 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.854087 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8k6wq" event={"ID":"6c605213-6328-49f7-b4e0-3a5283e3d27b","Type":"ContainerStarted","Data":"d319653e4aed6397efd5ea06b55df858802659e3b86e77c62417580f63af3173"} Jul 01 20:46:58.856754 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.856735 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" event={"ID":"2212862b-69c4-4886-a071-54cf439b9fd0","Type":"ContainerStarted","Data":"2182c429bca7c7dfbdd3e0d0fbbfc63880e335276ffa75c556b94e174acc0745"} Jul 01 20:46:58.859382 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.859355 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9k6kh" event={"ID":"c6a61d92-4e50-4935-9291-4a42de8fb80e","Type":"ContainerStarted","Data":"8627bbc44edf2bc2223bdf25bbc22e82d0c6e383e41895cacc9134287ace4578"} Jul 01 20:46:58.861770 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.861703 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-64tzn" event={"ID":"cf1d7d59-659b-43a0-ad53-cee5333fd5fe","Type":"ContainerStarted","Data":"ec032b9430afe933706efd7a039bc9db126222919e39355975b3f02efc83ee33"} Jul 01 20:46:58.864980 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.864957 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4k29z" event={"ID":"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa","Type":"ContainerStarted","Data":"86a1e950ea75d69596548198cb218397cbfe95b49815f335790f206a17b0d300"} Jul 01 20:46:58.893471 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.892771 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" event={"ID":"c3f741d9ca74dc13685dec19f2b0ff2d","Type":"ContainerStarted","Data":"6aa7830adae316c81d38b39bbe21c520282cf5a84877acf6e6a74c648304bc28"} Jul 01 20:46:58.907084 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.907048 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" event={"ID":"a06cf41c-de16-46c4-a552-ffb04f2bdbfe","Type":"ContainerStarted","Data":"dbd97d7d3d4eb1d22084562da02d6f61bfd2b886e6cfb211cc318a65ebe91051"} Jul 01 20:46:58.913524 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:58.913416 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"5c06363fbf9b2a10754a5ef2bb8591b9bcd63ec1fb6c6ab6efcb4ff9adcdcf28"} Jul 01 20:46:59.401145 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:59.400645 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:59.401145 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.400766 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:46:59.401145 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.400814 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs podName:220e4077-2a30-4706-863a-6ec30801b3f3 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:01.400801361 +0000 UTC m=+6.009244227 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs") pod "network-metrics-daemon-kkfpv" (UID: "220e4077-2a30-4706-863a-6ec30801b3f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:46:59.501519 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:59.501480 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:46:59.501702 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:59.501546 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:59.501702 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.501687 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 20:46:59.501806 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.501749 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret podName:1172c283-88c3-407e-8c99-4af696b5984c nodeName:}" failed. No retries permitted until 2026-07-01 20:47:01.501732105 +0000 UTC m=+6.110174965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret") pod "global-pull-secret-syncer-2ntmn" (UID: "1172c283-88c3-407e-8c99-4af696b5984c") : object "kube-system"/"original-pull-secret" not registered Jul 01 20:46:59.502168 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.502148 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 20:46:59.502251 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.502176 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 20:46:59.502251 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.502188 2558 projected.go:196] Error preparing data for projected volume kube-api-access-846pb for pod openshift-network-diagnostics/network-check-target-xj5kn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:46:59.502251 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.502230 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb podName:af091fbd-26ff-4648-9042-fa91ba493331 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:01.502216078 +0000 UTC m=+6.110658936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-846pb" (UniqueName: "kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb") pod "network-check-target-xj5kn" (UID: "af091fbd-26ff-4648-9042-fa91ba493331") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:46:59.841289 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:59.841175 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:46:59.841711 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.841323 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:46:59.841778 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:59.841731 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:46:59.841860 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.841837 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:46:59.841931 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:59.841910 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:46:59.842018 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:46:59.841977 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:46:59.842089 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:59.842065 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 20:41:57 +0000 UTC" deadline="2027-12-18 13:14:44.179776924 +0000 UTC" Jul 01 20:46:59.842143 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:46:59.842092 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12832h27m44.337688746s" Jul 01 20:47:01.420724 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:01.420688 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:01.421186 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.420868 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:47:01.421186 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.420942 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs podName:220e4077-2a30-4706-863a-6ec30801b3f3 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:05.42092068 +0000 UTC m=+10.029363555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs") pod "network-metrics-daemon-kkfpv" (UID: "220e4077-2a30-4706-863a-6ec30801b3f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:47:01.522036 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:01.521297 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:01.522036 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:01.521401 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:01.522036 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.521521 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 20:47:01.522036 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.521536 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 20:47:01.522036 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.521552 2558 projected.go:196] Error preparing data for projected volume kube-api-access-846pb for pod openshift-network-diagnostics/network-check-target-xj5kn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:47:01.522036 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.521611 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb podName:af091fbd-26ff-4648-9042-fa91ba493331 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:05.521592594 +0000 UTC m=+10.130035453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-846pb" (UniqueName: "kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb") pod "network-check-target-xj5kn" (UID: "af091fbd-26ff-4648-9042-fa91ba493331") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:47:01.522036 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.521955 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 20:47:01.522036 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.522002 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret podName:1172c283-88c3-407e-8c99-4af696b5984c nodeName:}" failed. No retries permitted until 2026-07-01 20:47:05.521987975 +0000 UTC m=+10.130430846 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret") pod "global-pull-secret-syncer-2ntmn" (UID: "1172c283-88c3-407e-8c99-4af696b5984c") : object "kube-system"/"original-pull-secret" not registered Jul 01 20:47:01.839434 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:01.839351 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:01.839582 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.839484 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:01.839906 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:01.839876 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:01.840036 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.839965 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:01.840102 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:01.840074 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:01.840156 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:01.840119 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:03.838566 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:03.838521 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:03.839033 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:03.838660 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:03.839033 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:03.838521 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:03.839033 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:03.838913 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:03.839033 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:03.838983 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:03.839207 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:03.839064 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:05.451402 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:05.451363 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:05.451782 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.451543 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:47:05.451782 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.451608 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs podName:220e4077-2a30-4706-863a-6ec30801b3f3 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:13.45159044 +0000 UTC m=+18.060033313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs") pod "network-metrics-daemon-kkfpv" (UID: "220e4077-2a30-4706-863a-6ec30801b3f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:47:05.551902 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:05.551866 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:05.552090 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:05.551923 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:05.552090 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.552037 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 20:47:05.552090 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.552040 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 20:47:05.552090 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.552066 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 20:47:05.552090 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.552080 2558 projected.go:196] Error preparing data for projected volume kube-api-access-846pb for pod openshift-network-diagnostics/network-check-target-xj5kn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:47:05.552334 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.552095 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret podName:1172c283-88c3-407e-8c99-4af696b5984c nodeName:}" failed. No retries permitted until 2026-07-01 20:47:13.552077086 +0000 UTC m=+18.160519959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret") pod "global-pull-secret-syncer-2ntmn" (UID: "1172c283-88c3-407e-8c99-4af696b5984c") : object "kube-system"/"original-pull-secret" not registered Jul 01 20:47:05.552334 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.552113 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb podName:af091fbd-26ff-4648-9042-fa91ba493331 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:13.55210267 +0000 UTC m=+18.160545523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-846pb" (UniqueName: "kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb") pod "network-check-target-xj5kn" (UID: "af091fbd-26ff-4648-9042-fa91ba493331") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:47:05.839429 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:05.839352 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:05.839593 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.839461 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:05.840130 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:05.839834 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:05.840130 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.839987 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:05.840130 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:05.840031 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:05.840130 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:05.840096 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:07.840629 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:07.840595 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:07.841015 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:07.840704 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:07.841084 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:07.841068 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:07.841172 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:07.841153 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:07.841229 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:07.841217 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:07.841327 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:07.841307 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:09.838293 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:09.838238 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:09.838747 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:09.838238 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:09.838747 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:09.838398 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:09.838747 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:09.838238 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:09.838747 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:09.838478 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:09.838747 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:09.838537 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:11.839064 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:11.839036 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:11.839064 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:11.839067 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:11.839645 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:11.839121 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:11.839645 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:11.839218 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:11.839645 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:11.839300 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:11.839645 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:11.839381 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:13.517524 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:13.517424 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:13.517942 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.517586 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:47:13.517942 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.517654 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs podName:220e4077-2a30-4706-863a-6ec30801b3f3 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:29.517640083 +0000 UTC m=+34.126082938 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs") pod "network-metrics-daemon-kkfpv" (UID: "220e4077-2a30-4706-863a-6ec30801b3f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:47:13.618200 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:13.618162 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:13.618403 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:13.618228 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:13.618403 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.618338 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 20:47:13.618403 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.618353 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 20:47:13.618403 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.618374 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 20:47:13.618403 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.618383 2558 projected.go:196] Error preparing data for projected volume kube-api-access-846pb for pod openshift-network-diagnostics/network-check-target-xj5kn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:47:13.618403 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.618398 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret podName:1172c283-88c3-407e-8c99-4af696b5984c nodeName:}" failed. No retries permitted until 2026-07-01 20:47:29.618382182 +0000 UTC m=+34.226825037 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret") pod "global-pull-secret-syncer-2ntmn" (UID: "1172c283-88c3-407e-8c99-4af696b5984c") : object "kube-system"/"original-pull-secret" not registered Jul 01 20:47:13.618769 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.618428 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb podName:af091fbd-26ff-4648-9042-fa91ba493331 nodeName:}" failed. No retries permitted until 2026-07-01 20:47:29.61841676 +0000 UTC m=+34.226859632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-846pb" (UniqueName: "kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb") pod "network-check-target-xj5kn" (UID: "af091fbd-26ff-4648-9042-fa91ba493331") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:47:13.838244 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:13.838157 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:13.838244 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:13.838197 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:13.838477 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.838308 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:13.838477 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:13.838331 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:13.838477 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.838426 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:13.838627 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:13.838518 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:15.838753 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:15.838720 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:15.839227 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:15.838828 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:15.839227 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:15.839177 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:15.839372 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:15.839294 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:15.839372 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:15.839350 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:15.839481 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:15.839417 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:17.839239 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.839071 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:17.839649 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.839071 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:17.839649 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.839131 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:17.839649 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:17.839386 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:17.839649 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:17.839502 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:17.839649 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:17.839598 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:17.960838 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.960800 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-64tzn" event={"ID":"cf1d7d59-659b-43a0-ad53-cee5333fd5fe","Type":"ContainerStarted","Data":"985f0eb777ddd2dc57f9b98b9b9c1229b4b9e2185bfdd8b07f0a444b6ca83568"} Jul 01 20:47:17.962390 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.962357 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4k29z" event={"ID":"98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa","Type":"ContainerStarted","Data":"b6b0801df41518da16c9a66cab4f509b1c93fe0ffd184cbcc746fd6df21db505"} Jul 01 20:47:17.966008 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.965984 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 20:47:17.966567 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.966541 2558 generic.go:356] "Generic (PLEG): container finished" podID="46646e21-8a7b-4443-98c5-5e5556d69ceb" containerID="76bc9d4708ed10901a9c10dc7917063d9ea45eaca65cfc94023a1e50c21bff32" exitCode=1 Jul 01 20:47:17.966674 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.966585 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"d0d9953c444b2c7bfafbd29d6f9f149372d89bfdbde573df1538cd5c0883900a"} Jul 01 20:47:17.966674 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.966654 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"44b97a5c42da425d5adbec27d6df5991283c5038ce8513de116a1601a50a0fab"} Jul 01 20:47:17.966776 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.966708 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"f65258f7ab05c9679d2af7c70fefc1d2e1664369247f33e3dcdc11f51fda2d39"} Jul 01 20:47:17.966776 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.966757 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"cda2e837f5acd62e758b04e6f24210205c10786727b0eadd3ae0ee37a4382a54"} Jul 01 20:47:17.966875 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.966807 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerDied","Data":"76bc9d4708ed10901a9c10dc7917063d9ea45eaca65cfc94023a1e50c21bff32"} Jul 01 20:47:17.966875 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.966854 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"01420230629d824be5b1096024d0cab617292a1e713eec6b02eb594bde768432"} Jul 01 20:47:17.992303 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.992226 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-64tzn" podStartSLOduration=3.761189048 podStartE2EDuration="22.992212567s" totalImagesPullingTime="19.231023519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:17.991992736 +0000 UTC m=+22.600435612" watchObservedRunningTime="2026-07-01 20:47:17.992212567 +0000 UTC m=+22.600655442" Jul 01 20:47:17.992558 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:17.992538 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-150.ec2.internal" podStartSLOduration=20.992532057 podStartE2EDuration="20.992532057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:46:58.913500658 +0000 UTC m=+3.521943638" watchObservedRunningTime="2026-07-01 20:47:17.992532057 +0000 UTC m=+22.600974932" Jul 01 20:47:18.970414 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.970157 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" event={"ID":"a06cf41c-de16-46c4-a552-ffb04f2bdbfe","Type":"ContainerStarted","Data":"e1058c5fa1d2bfa5bd7191ea571f0498b345dce543d6264f0f6bf277ccd02ff1"} Jul 01 20:47:18.971760 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.971736 2558 generic.go:356] "Generic (PLEG): container finished" podID="0316c5dcf0b57b896d9dfde967cd3a4e" containerID="cf997e561f98b5bb924c777baa3ae7208daa4361b7c8151dbda9554db33786b1" exitCode=0 Jul 01 20:47:18.971879 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.971811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" event={"ID":"0316c5dcf0b57b896d9dfde967cd3a4e","Type":"ContainerDied","Data":"cf997e561f98b5bb924c777baa3ae7208daa4361b7c8151dbda9554db33786b1"} Jul 01 20:47:18.973219 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.973194 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zmn82" event={"ID":"4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4","Type":"ContainerStarted","Data":"eab500bec2cbd03607e6bf0bd3a71bce159cca0a06e5bbce2b10caa4b2f58fe0"} Jul 01 20:47:18.974649 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.974629 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m5fg2" event={"ID":"e5b40ef0-7439-42ae-a63e-30d2c415238b","Type":"ContainerStarted","Data":"1c63c00323cb4d94f0ebb731eed4945e8f3f69aee6dd274d7bfb9cba24b361f5"} Jul 01 20:47:18.978691 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.978661 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8k6wq" event={"ID":"6c605213-6328-49f7-b4e0-3a5283e3d27b","Type":"ContainerStarted","Data":"c1a294e7adfa83c3654bf16292a4da11997448272bdc58fd7cca94d494c3710b"} Jul 01 20:47:18.980599 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.980576 2558 generic.go:356] "Generic (PLEG): container finished" podID="2212862b-69c4-4886-a071-54cf439b9fd0" containerID="591d29869a89d977419049fd89708b4b71ed4fd1795d5da6b792387ed1f91034" exitCode=0 Jul 01 20:47:18.980684 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.980662 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" event={"ID":"2212862b-69c4-4886-a071-54cf439b9fd0","Type":"ContainerDied","Data":"591d29869a89d977419049fd89708b4b71ed4fd1795d5da6b792387ed1f91034"} Jul 01 20:47:18.982067 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.982047 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9k6kh" event={"ID":"c6a61d92-4e50-4935-9291-4a42de8fb80e","Type":"ContainerStarted","Data":"4e86398633a7c1f0bf5a9ee5f8d8218b29cd7fea4900f1a1dc1fd9b92f0bab78"} Jul 01 20:47:18.989464 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:18.989420 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4k29z" podStartSLOduration=4.048481875 podStartE2EDuration="22.989402924s" totalImagesPullingTime="18.940921049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:18.031890374 +0000 UTC m=+22.640333250" watchObservedRunningTime="2026-07-01 20:47:18.989402924 +0000 UTC m=+23.597845816" Jul 01 20:47:19.025414 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:19.025369 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9k6kh" podStartSLOduration=4.093610537 podStartE2EDuration="23.025353704s" totalImagesPullingTime="18.931743167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:19.009750877 +0000 UTC m=+23.618193756" watchObservedRunningTime="2026-07-01 20:47:19.025353704 +0000 UTC m=+23.633796579" Jul 01 20:47:19.048361 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:19.048322 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m5fg2" podStartSLOduration=5.118399285 podStartE2EDuration="24.048307383s" totalImagesPullingTime="18.929908098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:19.025984134 +0000 UTC m=+23.634427014" watchObservedRunningTime="2026-07-01 20:47:19.048307383 +0000 UTC m=+23.656750257" Jul 01 20:47:19.083512 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:19.083472 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zmn82" podStartSLOduration=4.238981231 podStartE2EDuration="23.083457899s" totalImagesPullingTime="18.844476668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:19.083050939 +0000 UTC m=+23.691493813" watchObservedRunningTime="2026-07-01 20:47:19.083457899 +0000 UTC m=+23.691900775" Jul 01 20:47:19.083683 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:19.083662 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8k6wq" podStartSLOduration=4.148761025 podStartE2EDuration="23.083655761s" totalImagesPullingTime="18.934894736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:19.065873084 +0000 UTC m=+23.674315955" watchObservedRunningTime="2026-07-01 20:47:19.083655761 +0000 UTC m=+23.692098636" Jul 01 20:47:19.839037 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:19.839006 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:19.839232 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:19.839006 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:19.839232 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:19.839137 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:19.839232 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:19.839160 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:19.839365 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:19.839278 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:19.839365 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:19.839353 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:19.987088 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:19.987044 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" event={"ID":"0316c5dcf0b57b896d9dfde967cd3a4e","Type":"ContainerStarted","Data":"bc7a8fc9426627f09939734ed767aee9675e6804ce7dd789ab0e24f74210882f"} Jul 01 20:47:20.014846 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.014699 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-150.ec2.internal" podStartSLOduration=23.014678102 podStartE2EDuration="23.014678102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:20.014333701 +0000 UTC m=+24.622776578" watchObservedRunningTime="2026-07-01 20:47:20.014678102 +0000 UTC m=+24.623120978" Jul 01 20:47:20.163022 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.162996 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:47:20.163678 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.163658 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:47:20.289131 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.289105 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 20:47:20.834516 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.834389 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T20:47:20.289127538Z","UUID":"8ee9005b-9f47-42eb-b61d-db9fac5d9a66","Handler":null,"Name":"","Endpoint":""} Jul 01 20:47:20.836331 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.836307 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 20:47:20.836468 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.836343 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 20:47:20.992400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.992362 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" event={"ID":"a06cf41c-de16-46c4-a552-ffb04f2bdbfe","Type":"ContainerStarted","Data":"960403bc5b229723a4d000fd4a0017b8cc77125de642d62c1af0d3cfdc937590"} Jul 01 20:47:20.995924 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.995898 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 20:47:20.996428 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.996344 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"5f02fb9fa5cbed6cb76fafcfb1833d5b903bc16cfe36c80ff51b7b45cde65809"} Jul 01 20:47:20.996714 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.996681 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:47:20.997363 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:20.997138 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zmn82" Jul 01 20:47:21.838174 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:21.838142 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:21.838356 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:21.838145 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:21.838356 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:21.838286 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:21.838356 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:21.838149 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:21.838529 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:21.838367 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:21.838529 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:21.838441 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:22.003241 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:22.003202 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" event={"ID":"a06cf41c-de16-46c4-a552-ffb04f2bdbfe","Type":"ContainerStarted","Data":"3d19f6871cfeaf5112eed48951e995d4d702c2b7cc18f7b7f5b528d093bd772b"} Jul 01 20:47:22.027325 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:22.027278 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9rjlz" podStartSLOduration=4.531788673 podStartE2EDuration="26.027250704s" totalImagesPullingTime="21.495462031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:22.026920623 +0000 UTC m=+26.635363498" watchObservedRunningTime="2026-07-01 20:47:22.027250704 +0000 UTC m=+26.635693579" Jul 01 20:47:23.010848 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.010388 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 20:47:23.012204 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.011703 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"ba3e5e5a387bb1f9c0b8623adb62b833b41003b325a2d4d255c59e1118709739"} Jul 01 20:47:23.012620 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.012589 2558 scope.go:117] "RemoveContainer" containerID="76bc9d4708ed10901a9c10dc7917063d9ea45eaca65cfc94023a1e50c21bff32" Jul 01 20:47:23.012620 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.012600 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:47:23.012757 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.012699 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:47:23.012757 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.012751 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:47:23.037339 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.037199 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:47:23.039162 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.039141 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:47:23.839095 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.839067 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:23.839398 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.839067 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:23.839398 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:23.839158 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:23.839398 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:23.839175 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:23.839398 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:23.839250 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:23.839398 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:23.839329 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:24.014963 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:24.014930 2558 generic.go:356] "Generic (PLEG): container finished" podID="2212862b-69c4-4886-a071-54cf439b9fd0" containerID="73aab1b93cc6be9783b018c9b1570233107f5d82102e3e7ba10fec65bea4412a" exitCode=0 Jul 01 20:47:24.015674 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:24.014985 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" event={"ID":"2212862b-69c4-4886-a071-54cf439b9fd0","Type":"ContainerDied","Data":"73aab1b93cc6be9783b018c9b1570233107f5d82102e3e7ba10fec65bea4412a"} Jul 01 20:47:24.018969 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:24.018945 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 20:47:24.019397 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:24.019376 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" event={"ID":"46646e21-8a7b-4443-98c5-5e5556d69ceb","Type":"ContainerStarted","Data":"6728d6b04052e7ae46912deb999c269501e509f9f3e1d54ebaf57cfd5c743156"} Jul 01 20:47:25.023320 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:25.023219 2558 generic.go:356] "Generic (PLEG): container finished" podID="2212862b-69c4-4886-a071-54cf439b9fd0" containerID="af302efad3a8e4379dcb0086aac0c0d2fe8448ace3a2390129585aa6c73ff92e" exitCode=0 Jul 01 20:47:25.023710 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:25.023312 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" event={"ID":"2212862b-69c4-4886-a071-54cf439b9fd0","Type":"ContainerDied","Data":"af302efad3a8e4379dcb0086aac0c0d2fe8448ace3a2390129585aa6c73ff92e"} Jul 01 20:47:25.054176 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:25.054133 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" podStartSLOduration=10.091738336 podStartE2EDuration="29.054118341s" totalImagesPullingTime="18.962380005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:24.082431598 +0000 UTC m=+28.690874473" watchObservedRunningTime="2026-07-01 20:47:25.054118341 +0000 UTC m=+29.662561195" Jul 01 20:47:25.839297 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:25.839246 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:25.839481 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:25.839347 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:25.839481 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:25.839360 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:25.839481 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:25.839386 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:25.839481 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:25.839461 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:25.839695 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:25.839514 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:26.028486 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:26.028235 2558 generic.go:356] "Generic (PLEG): container finished" podID="2212862b-69c4-4886-a071-54cf439b9fd0" containerID="a140e86a7aafbe710d06a6c86f097ed3ab93882b8957b66caa85b7b3e40957cc" exitCode=0 Jul 01 20:47:26.028486 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:26.028292 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" event={"ID":"2212862b-69c4-4886-a071-54cf439b9fd0","Type":"ContainerDied","Data":"a140e86a7aafbe710d06a6c86f097ed3ab93882b8957b66caa85b7b3e40957cc"} Jul 01 20:47:27.838653 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:27.838620 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:27.838653 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:27.838633 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:27.839155 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:27.838621 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:27.839155 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:27.838749 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:27.839155 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:27.838825 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:27.839155 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:27.838908 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:29.543205 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:29.542678 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:29.543205 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.542831 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:47:29.543205 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.542892 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs podName:220e4077-2a30-4706-863a-6ec30801b3f3 nodeName:}" failed. No retries permitted until 2026-07-01 20:48:01.542874164 +0000 UTC m=+66.151317030 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs") pod "network-metrics-daemon-kkfpv" (UID: "220e4077-2a30-4706-863a-6ec30801b3f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 20:47:29.643935 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:29.643901 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:29.644098 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:29.643960 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:29.644098 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.644048 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 20:47:29.644098 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.644067 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 20:47:29.644098 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.644079 2558 projected.go:196] Error preparing data for projected volume kube-api-access-846pb for pod openshift-network-diagnostics/network-check-target-xj5kn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:47:29.644098 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.644080 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 20:47:29.644340 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.644138 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb podName:af091fbd-26ff-4648-9042-fa91ba493331 nodeName:}" failed. No retries permitted until 2026-07-01 20:48:01.644121183 +0000 UTC m=+66.252564044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-846pb" (UniqueName: "kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb") pod "network-check-target-xj5kn" (UID: "af091fbd-26ff-4648-9042-fa91ba493331") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 20:47:29.644340 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.644158 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret podName:1172c283-88c3-407e-8c99-4af696b5984c nodeName:}" failed. No retries permitted until 2026-07-01 20:48:01.644151155 +0000 UTC m=+66.252594012 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret") pod "global-pull-secret-syncer-2ntmn" (UID: "1172c283-88c3-407e-8c99-4af696b5984c") : object "kube-system"/"original-pull-secret" not registered Jul 01 20:47:29.838912 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:29.838839 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:29.839062 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:29.838839 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:29.839062 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:29.838963 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:29.839062 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.838958 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:29.839062 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.839035 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:29.839293 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:29.839131 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:30.494487 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:30.494384 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2ntmn"] Jul 01 20:47:30.494897 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:30.494751 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:30.501949 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:30.498325 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kkfpv"] Jul 01 20:47:30.501949 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:30.498400 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:30.501949 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:30.498909 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xj5kn"] Jul 01 20:47:30.501949 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:30.499017 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:30.501949 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:30.499177 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:30.501949 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:30.499302 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:30.502571 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:30.502544 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:31.043291 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:31.043234 2558 generic.go:356] "Generic (PLEG): container finished" podID="2212862b-69c4-4886-a071-54cf439b9fd0" containerID="5c9eb64ecb90ae1272cf87f4111f7598f4a2770d01f095d807d8c7c3b8d2fab7" exitCode=0 Jul 01 20:47:31.043692 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:31.043323 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" event={"ID":"2212862b-69c4-4886-a071-54cf439b9fd0","Type":"ContainerDied","Data":"5c9eb64ecb90ae1272cf87f4111f7598f4a2770d01f095d807d8c7c3b8d2fab7"} Jul 01 20:47:31.838813 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:31.838777 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:31.838999 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:31.838893 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:32.048776 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:32.048738 2558 generic.go:356] "Generic (PLEG): container finished" podID="2212862b-69c4-4886-a071-54cf439b9fd0" containerID="0b061af7c6b94ce22d05a5e5f15b3d643584a3522c9715b38c4beec046de2baa" exitCode=0 Jul 01 20:47:32.049228 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:32.048787 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" event={"ID":"2212862b-69c4-4886-a071-54cf439b9fd0","Type":"ContainerDied","Data":"0b061af7c6b94ce22d05a5e5f15b3d643584a3522c9715b38c4beec046de2baa"} Jul 01 20:47:32.838787 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:32.838760 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:32.838787 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:32.838792 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:32.838974 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:32.838869 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:32.839018 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:32.838993 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:33.056958 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:33.056927 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" event={"ID":"2212862b-69c4-4886-a071-54cf439b9fd0","Type":"ContainerStarted","Data":"efd2c7aa834615aa55fc8c8a26538715f1e0f869fea3448ba07e07e3147334e3"} Jul 01 20:47:33.095044 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:33.094923 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8wkrd" podStartSLOduration=9.65088723 podStartE2EDuration="38.09490519s" totalImagesPullingTime="28.44401796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:33.094644293 +0000 UTC m=+37.703087173" watchObservedRunningTime="2026-07-01 20:47:33.09490519 +0000 UTC m=+37.703348066" Jul 01 20:47:33.838619 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:33.838590 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:33.838757 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:33.838707 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkfpv" podUID="220e4077-2a30-4706-863a-6ec30801b3f3" Jul 01 20:47:34.838455 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:34.838419 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:34.838867 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:34.838548 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2ntmn" podUID="1172c283-88c3-407e-8c99-4af696b5984c" Jul 01 20:47:34.838867 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:34.838593 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:34.838867 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:47:34.838677 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj5kn" podUID="af091fbd-26ff-4648-9042-fa91ba493331" Jul 01 20:47:35.203429 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.203345 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-150.ec2.internal" event="NodeReady" Jul 01 20:47:35.203570 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.203494 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 20:47:35.283187 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.283156 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7996c789b7-qvjxp"] Jul 01 20:47:35.298321 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.298301 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.302985 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.302954 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 20:47:35.302985 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.302967 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 20:47:35.303136 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.302962 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 20:47:35.303383 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.303369 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pw9xl\"" Jul 01 20:47:35.309740 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.309721 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 20:47:35.345184 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.345147 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7996c789b7-qvjxp"] Jul 01 20:47:35.345719 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.345698 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-85h8x"] Jul 01 20:47:35.354646 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.354626 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.357620 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.357603 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 20:47:35.357689 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.357669 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 20:47:35.360731 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.358485 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tfxg4\"" Jul 01 20:47:35.372602 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.372577 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-h4t7s"] Jul 01 20:47:35.393798 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.393777 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-85h8x"] Jul 01 20:47:35.393906 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.393858 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h4t7s" Jul 01 20:47:35.400060 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.400043 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 20:47:35.400160 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.400141 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 20:47:35.400217 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.400141 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vpvmm\"" Jul 01 20:47:35.406121 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.406099 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 20:47:35.413506 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.413478 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h4t7s"] Jul 01 20:47:35.432677 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.432658 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dc864"] Jul 01 20:47:35.466898 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.466830 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dc864"] Jul 01 20:47:35.467015 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.466943 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.472951 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.472929 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 20:47:35.473080 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.472932 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 20:47:35.473080 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.472933 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j9f8h\"" Jul 01 20:47:35.473250 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.473110 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 20:47:35.473355 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.473341 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 20:47:35.485294 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485272 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-registry-certificates\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.485400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485305 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65p64\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-kube-api-access-65p64\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.485400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485324 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dm58\" (UniqueName: \"kubernetes.io/projected/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-kube-api-access-8dm58\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.485400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485350 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-metrics-tls\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.485551 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485411 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-trusted-ca\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.485551 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485437 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-ca-trust-extracted\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.485551 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485455 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-installation-pull-secrets\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.485551 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485501 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-config-volume\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.485699 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485565 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-tmp-dir\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.485699 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485590 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-image-registry-private-configuration\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.485699 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485613 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-registry-tls\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.485699 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.485627 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-bound-sa-token\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.586850 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.586812 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-trusted-ca\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.586997 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.586863 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-ca-trust-extracted\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.586997 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.586887 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-installation-pull-secrets\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.586997 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.586905 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-config-volume\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.586997 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.586923 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5117067-8f06-466b-aa57-be171670744e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.586997 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.586939 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp8mf\" (UniqueName: \"kubernetes.io/projected/b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629-kube-api-access-pp8mf\") pod \"ingress-canary-h4t7s\" (UID: \"b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629\") " pod="openshift-ingress-canary/ingress-canary-h4t7s" Jul 01 20:47:35.586997 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.586963 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-tmp-dir\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.587314 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587006 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5117067-8f06-466b-aa57-be171670744e-data-volume\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.587314 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587051 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-image-registry-private-configuration\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.587314 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629-cert\") pod \"ingress-canary-h4t7s\" (UID: \"b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629\") " pod="openshift-ingress-canary/ingress-canary-h4t7s" Jul 01 20:47:35.587314 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587115 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-registry-tls\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.587314 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587227 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-bound-sa-token\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.587534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587329 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-ca-trust-extracted\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.587534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587390 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5117067-8f06-466b-aa57-be171670744e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.587534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587393 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-tmp-dir\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.587534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587418 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7ftl\" (UniqueName: \"kubernetes.io/projected/e5117067-8f06-466b-aa57-be171670744e-kube-api-access-p7ftl\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.587534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587475 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-registry-certificates\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.587534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587495 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65p64\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-kube-api-access-65p64\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.587534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587516 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dm58\" (UniqueName: \"kubernetes.io/projected/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-kube-api-access-8dm58\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.587860 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587549 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-metrics-tls\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.587860 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587572 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5117067-8f06-466b-aa57-be171670744e-crio-socket\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.587860 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587599 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-config-volume\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.588021 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.587983 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-trusted-ca\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.588188 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.588164 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-registry-certificates\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.591362 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.591343 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-registry-tls\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.591442 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.591346 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-metrics-tls\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.591442 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.591415 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-installation-pull-secrets\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.591510 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.591416 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-image-registry-private-configuration\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.616855 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.616820 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dm58\" (UniqueName: \"kubernetes.io/projected/0ef76ab8-0b86-4d6c-bfb9-965f0fc84219-kube-api-access-8dm58\") pod \"dns-default-85h8x\" (UID: \"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219\") " pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.617064 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.617050 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65p64\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-kube-api-access-65p64\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.618110 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.618095 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7-bound-sa-token\") pod \"image-registry-7996c789b7-qvjxp\" (UID: \"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7\") " pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.665277 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.665232 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:35.688121 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688090 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7ftl\" (UniqueName: \"kubernetes.io/projected/e5117067-8f06-466b-aa57-be171670744e-kube-api-access-p7ftl\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.688253 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688156 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5117067-8f06-466b-aa57-be171670744e-crio-socket\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.688344 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688328 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5117067-8f06-466b-aa57-be171670744e-crio-socket\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.688406 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688348 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5117067-8f06-466b-aa57-be171670744e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.688406 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp8mf\" (UniqueName: \"kubernetes.io/projected/b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629-kube-api-access-pp8mf\") pod \"ingress-canary-h4t7s\" (UID: \"b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629\") " pod="openshift-ingress-canary/ingress-canary-h4t7s" Jul 01 20:47:35.688406 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688400 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5117067-8f06-466b-aa57-be171670744e-data-volume\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.688544 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688421 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629-cert\") pod \"ingress-canary-h4t7s\" (UID: \"b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629\") " pod="openshift-ingress-canary/ingress-canary-h4t7s" Jul 01 20:47:35.688544 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688457 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5117067-8f06-466b-aa57-be171670744e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.688843 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688813 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5117067-8f06-466b-aa57-be171670744e-data-volume\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.688954 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.688928 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5117067-8f06-466b-aa57-be171670744e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.690982 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.690962 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5117067-8f06-466b-aa57-be171670744e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.691294 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.691277 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629-cert\") pod \"ingress-canary-h4t7s\" (UID: \"b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629\") " pod="openshift-ingress-canary/ingress-canary-h4t7s" Jul 01 20:47:35.713653 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.713611 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp8mf\" (UniqueName: \"kubernetes.io/projected/b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629-kube-api-access-pp8mf\") pod \"ingress-canary-h4t7s\" (UID: \"b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629\") " pod="openshift-ingress-canary/ingress-canary-h4t7s" Jul 01 20:47:35.714955 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.714928 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7ftl\" (UniqueName: \"kubernetes.io/projected/e5117067-8f06-466b-aa57-be171670744e-kube-api-access-p7ftl\") pod \"insights-runtime-extractor-dc864\" (UID: \"e5117067-8f06-466b-aa57-be171670744e\") " pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.775563 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.775535 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dc864" Jul 01 20:47:35.841711 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.841680 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:47:35.850346 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.850321 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 20:47:35.850460 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.850414 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7lgrj\"" Jul 01 20:47:35.908087 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.908058 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:35.921751 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.921717 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-85h8x"] Jul 01 20:47:35.930491 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:47:35.930451 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef76ab8_0b86_4d6c_bfb9_965f0fc84219.slice/crio-456348bafee814b39098ff6391e38f07f0f497abf5186189aedfb87daa6bfa65 WatchSource:0}: Error finding container 456348bafee814b39098ff6391e38f07f0f497abf5186189aedfb87daa6bfa65: Status 404 returned error can't find the container with id 456348bafee814b39098ff6391e38f07f0f497abf5186189aedfb87daa6bfa65 Jul 01 20:47:35.988835 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:35.988752 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dc864"] Jul 01 20:47:36.001671 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.001645 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h4t7s" Jul 01 20:47:36.069216 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.068558 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dc864" event={"ID":"e5117067-8f06-466b-aa57-be171670744e","Type":"ContainerStarted","Data":"b1312f6b194a106eb20d1ad12b4d2b1de334ca39d7d4bf7cd35f325d1ac66801"} Jul 01 20:47:36.070079 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.070053 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85h8x" event={"ID":"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219","Type":"ContainerStarted","Data":"456348bafee814b39098ff6391e38f07f0f497abf5186189aedfb87daa6bfa65"} Jul 01 20:47:36.083381 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.082935 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7996c789b7-qvjxp"] Jul 01 20:47:36.087211 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:47:36.087188 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6fc79e_6522_45bf_9e7c_387c1d5ea1f7.slice/crio-c3d5be00c0bbee0a986b96a9b72fcc9c8d37da400097f988132e89b2a09c1ad5 WatchSource:0}: Error finding container c3d5be00c0bbee0a986b96a9b72fcc9c8d37da400097f988132e89b2a09c1ad5: Status 404 returned error can't find the container with id c3d5be00c0bbee0a986b96a9b72fcc9c8d37da400097f988132e89b2a09c1ad5 Jul 01 20:47:36.166895 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.166860 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h4t7s"] Jul 01 20:47:36.175335 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:47:36.175301 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0ab5d3e_ed68_4d5e_8bf8_a82ee19bb629.slice/crio-7af324151b227c71b468aa49bbef6bfcd3470aad394706b3ed74fe56aa7f428f WatchSource:0}: Error finding container 7af324151b227c71b468aa49bbef6bfcd3470aad394706b3ed74fe56aa7f428f: Status 404 returned error can't find the container with id 7af324151b227c71b468aa49bbef6bfcd3470aad394706b3ed74fe56aa7f428f Jul 01 20:47:36.839905 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.838913 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:47:36.839905 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.839345 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:47:36.844325 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.844142 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 20:47:36.844704 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.844447 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r694q\"" Jul 01 20:47:36.846193 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.846006 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 20:47:36.847093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:36.846910 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 20:47:37.076705 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:37.076665 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h4t7s" event={"ID":"b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629","Type":"ContainerStarted","Data":"7af324151b227c71b468aa49bbef6bfcd3470aad394706b3ed74fe56aa7f428f"} Jul 01 20:47:37.079786 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:37.079721 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" event={"ID":"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7","Type":"ContainerStarted","Data":"c92e2824330389f55cae3422f5627e6afa737a9213a53452b28d57b7eb8f926c"} Jul 01 20:47:37.079786 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:37.079778 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" event={"ID":"9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7","Type":"ContainerStarted","Data":"c3d5be00c0bbee0a986b96a9b72fcc9c8d37da400097f988132e89b2a09c1ad5"} Jul 01 20:47:37.080632 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:37.080607 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:37.115464 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:37.115359 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" podStartSLOduration=8.11533597 podStartE2EDuration="8.11533597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:37.114229634 +0000 UTC m=+41.722672510" watchObservedRunningTime="2026-07-01 20:47:37.11533597 +0000 UTC m=+41.723778846" Jul 01 20:47:41.149573 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.149539 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2fvf4"] Jul 01 20:47:41.223188 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.223155 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.225921 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.225889 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 20:47:41.226225 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.226203 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 20:47:41.228236 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.226975 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 20:47:41.228236 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.227461 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v8blg\"" Jul 01 20:47:41.228236 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.227633 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 20:47:41.228236 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.227773 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 20:47:41.228236 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.227888 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 20:47:41.233554 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233532 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-tls\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.233631 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233578 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftdxg\" (UniqueName: \"kubernetes.io/projected/38dfda6f-4326-44d6-a494-3d40212210c6-kube-api-access-ftdxg\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.233677 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233629 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-wtmp\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.233721 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233683 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-sys\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.233721 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233712 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.233810 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233738 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38dfda6f-4326-44d6-a494-3d40212210c6-metrics-client-ca\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.233810 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233769 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-textfile\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.233810 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233797 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.233921 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.233909 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-root\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334195 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334163 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-root\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334195 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334198 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-tls\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334219 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftdxg\" (UniqueName: \"kubernetes.io/projected/38dfda6f-4326-44d6-a494-3d40212210c6-kube-api-access-ftdxg\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334279 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-wtmp\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334303 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-root\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334334 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-sys\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334361 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334383 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38dfda6f-4326-44d6-a494-3d40212210c6-metrics-client-ca\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334409 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-textfile\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334441 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-wtmp\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334810 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334626 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38dfda6f-4326-44d6-a494-3d40212210c6-sys\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.334916 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.334898 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-textfile\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.335046 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.335019 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.335116 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.335023 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38dfda6f-4326-44d6-a494-3d40212210c6-metrics-client-ca\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.336948 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.336925 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-tls\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.337095 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.337065 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38dfda6f-4326-44d6-a494-3d40212210c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.346409 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.346387 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftdxg\" (UniqueName: \"kubernetes.io/projected/38dfda6f-4326-44d6-a494-3d40212210c6-kube-api-access-ftdxg\") pod \"node-exporter-2fvf4\" (UID: \"38dfda6f-4326-44d6-a494-3d40212210c6\") " pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.534845 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.534764 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2fvf4" Jul 01 20:47:41.762299 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.761494 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64dcd6fc8c-9vsqd"] Jul 01 20:47:41.793505 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.793433 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64dcd6fc8c-9vsqd"] Jul 01 20:47:41.793650 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.793582 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.796379 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.796186 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 01 20:47:41.796379 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.796191 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-t872r\"" Jul 01 20:47:41.796379 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.796245 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 01 20:47:41.796379 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.796193 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 01 20:47:41.796591 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.796524 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 01 20:47:41.796824 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.796804 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 01 20:47:41.796944 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.796843 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 01 20:47:41.799575 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.799496 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 01 20:47:41.802320 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.802302 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 01 20:47:41.838605 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.838566 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-oauth-config\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.838788 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.838613 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn2sx\" (UniqueName: \"kubernetes.io/projected/42aaee9d-0478-4478-9a7b-afe423df82f6-kube-api-access-rn2sx\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.838788 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.838659 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-service-ca\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.838788 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.838708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-oauth-serving-cert\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.838788 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.838769 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-serving-cert\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.839005 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.838797 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-trusted-ca-bundle\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.839005 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.838841 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-console-config\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.940163 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.940127 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-serving-cert\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.940163 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.940162 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-trusted-ca-bundle\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.940416 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.940212 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-console-config\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.940416 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.940239 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-oauth-config\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.940416 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.940282 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn2sx\" (UniqueName: \"kubernetes.io/projected/42aaee9d-0478-4478-9a7b-afe423df82f6-kube-api-access-rn2sx\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.940564 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.940428 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-service-ca\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.940564 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.940477 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-oauth-serving-cert\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.941020 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.940997 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-console-config\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.941144 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.941127 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-oauth-serving-cert\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.941393 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.941372 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-service-ca\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.941485 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.941466 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-trusted-ca-bundle\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.942800 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.942775 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-serving-cert\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.942898 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.942883 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-oauth-config\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.951991 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:41.951964 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn2sx\" (UniqueName: \"kubernetes.io/projected/42aaee9d-0478-4478-9a7b-afe423df82f6-kube-api-access-rn2sx\") pod \"console-64dcd6fc8c-9vsqd\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:41.952101 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:47:41.952083 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dfda6f_4326_44d6_a494_3d40212210c6.slice/crio-7613ab8dd1edec034f6dd5a5e9ce6a04ccda553e552c6a3ef5119ed8aef65782 WatchSource:0}: Error finding container 7613ab8dd1edec034f6dd5a5e9ce6a04ccda553e552c6a3ef5119ed8aef65782: Status 404 returned error can't find the container with id 7613ab8dd1edec034f6dd5a5e9ce6a04ccda553e552c6a3ef5119ed8aef65782 Jul 01 20:47:42.094721 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:42.094685 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h4t7s" event={"ID":"b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629","Type":"ContainerStarted","Data":"0466953ea1eda9b8924023a6aefef3dd1642af14deeb8a26a34124db1390c53a"} Jul 01 20:47:42.096124 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:42.096091 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85h8x" event={"ID":"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219","Type":"ContainerStarted","Data":"ac48de9592c3af68e10661f629541a48049ed1f02a9a78e837dadb2bab5049f1"} Jul 01 20:47:42.097325 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:42.097301 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fvf4" event={"ID":"38dfda6f-4326-44d6-a494-3d40212210c6","Type":"ContainerStarted","Data":"7613ab8dd1edec034f6dd5a5e9ce6a04ccda553e552c6a3ef5119ed8aef65782"} Jul 01 20:47:42.105653 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:42.105620 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:42.279094 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:42.279060 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64dcd6fc8c-9vsqd"] Jul 01 20:47:42.433104 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:47:42.433065 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42aaee9d_0478_4478_9a7b_afe423df82f6.slice/crio-b58454d1dceed840bc818aedbd2b292547f86fe172509589d1b2f0debf996109 WatchSource:0}: Error finding container b58454d1dceed840bc818aedbd2b292547f86fe172509589d1b2f0debf996109: Status 404 returned error can't find the container with id b58454d1dceed840bc818aedbd2b292547f86fe172509589d1b2f0debf996109 Jul 01 20:47:43.103833 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:43.103785 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85h8x" event={"ID":"0ef76ab8-0b86-4d6c-bfb9-965f0fc84219","Type":"ContainerStarted","Data":"c8b3672c54e986c6d897e67dc223d9f12497cfc0501d5a2af37fb5a0ff20aab7"} Jul 01 20:47:43.104759 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:43.104728 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:43.106656 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:43.106621 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dc864" event={"ID":"e5117067-8f06-466b-aa57-be171670744e","Type":"ContainerStarted","Data":"f724310d203eebe1a2fe56260f2e86d9619216df64265d9b6f8ff10ed283f8be"} Jul 01 20:47:43.108437 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:43.108407 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dcd6fc8c-9vsqd" event={"ID":"42aaee9d-0478-4478-9a7b-afe423df82f6","Type":"ContainerStarted","Data":"b58454d1dceed840bc818aedbd2b292547f86fe172509589d1b2f0debf996109"} Jul 01 20:47:43.134028 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:43.133963 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-85h8x" podStartSLOduration=2.119886059 podStartE2EDuration="8.133938399s" totalImagesPullingTime="6.01405234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:43.132099796 +0000 UTC m=+47.740542672" watchObservedRunningTime="2026-07-01 20:47:43.133938399 +0000 UTC m=+47.742381275" Jul 01 20:47:43.157806 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:43.157747 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-h4t7s" podStartSLOduration=2.387750138 podStartE2EDuration="8.157730408s" totalImagesPullingTime="5.76998027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:43.156735826 +0000 UTC m=+47.765178702" watchObservedRunningTime="2026-07-01 20:47:43.157730408 +0000 UTC m=+47.766173284" Jul 01 20:47:44.114497 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.114468 2558 generic.go:356] "Generic (PLEG): container finished" podID="38dfda6f-4326-44d6-a494-3d40212210c6" containerID="cb4f10ec62be2f2607171d783e84cc21ee18ab05adf5cb10c5812c57132ec4ab" exitCode=0 Jul 01 20:47:44.114934 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.114912 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fvf4" event={"ID":"38dfda6f-4326-44d6-a494-3d40212210c6","Type":"ContainerDied","Data":"cb4f10ec62be2f2607171d783e84cc21ee18ab05adf5cb10c5812c57132ec4ab"} Jul 01 20:47:44.387165 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.387130 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5694969698-8zwdm"] Jul 01 20:47:44.390675 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.390658 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.394650 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.394627 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 01 20:47:44.394907 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.394870 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 01 20:47:44.395099 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.394906 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 01 20:47:44.395099 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.394909 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r2vdt\"" Jul 01 20:47:44.395740 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.395560 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 01 20:47:44.395740 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.395645 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1939ukfgambo6\"" Jul 01 20:47:44.395740 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.395657 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 01 20:47:44.412906 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.412873 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5694969698-8zwdm"] Jul 01 20:47:44.461581 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.461547 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-metrics-client-ca\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.461581 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.461594 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.461837 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.461613 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.461837 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.461694 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.461837 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.461734 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-grpc-tls\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.461837 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.461763 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.461837 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.461808 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwffs\" (UniqueName: \"kubernetes.io/projected/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-kube-api-access-jwffs\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.462030 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.461947 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-tls\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.563096 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.563062 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-tls\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.563296 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.563117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-metrics-client-ca\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.563296 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.563158 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.563296 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.563184 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.563296 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.563209 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.563296 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.563230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-grpc-tls\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.563296 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.563288 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.563551 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.563332 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwffs\" (UniqueName: \"kubernetes.io/projected/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-kube-api-access-jwffs\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.564750 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.564458 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-metrics-client-ca\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.566871 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.566845 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.567369 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.567346 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.567369 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.567358 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-tls\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.567504 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.567393 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-grpc-tls\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.567504 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.567427 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.568311 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.568289 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.576943 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.576918 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwffs\" (UniqueName: \"kubernetes.io/projected/a64e2477-7f8f-4ed4-b0bd-51f6f2f31252-kube-api-access-jwffs\") pod \"thanos-querier-5694969698-8zwdm\" (UID: \"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252\") " pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:44.702506 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:44.702466 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:45.120245 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.120206 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dc864" event={"ID":"e5117067-8f06-466b-aa57-be171670744e","Type":"ContainerStarted","Data":"56e00ccb2ac926d98b561398a31541b43a28c846c4d355a7502fa82cf935d05f"} Jul 01 20:47:45.122773 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.122677 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fvf4" event={"ID":"38dfda6f-4326-44d6-a494-3d40212210c6","Type":"ContainerStarted","Data":"60372464825ec0a274c335c45177fe70cf81ac2be12ffd06688965fcef40ce24"} Jul 01 20:47:45.122773 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.122745 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fvf4" event={"ID":"38dfda6f-4326-44d6-a494-3d40212210c6","Type":"ContainerStarted","Data":"d0ee8556a6ceb54dad5188099b75f62e5fd326655e618693b459a1ff61a4b7ef"} Jul 01 20:47:45.144830 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.144781 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2fvf4" podStartSLOduration=2.316055756 podStartE2EDuration="4.144763637s" totalImagesPullingTime="1.723832261s" totalInitContainerRuntime="104.87562ms" isStatefulPod=true podCreationTimestamp="2026-07-01 20:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:45.143224278 +0000 UTC m=+49.751667154" watchObservedRunningTime="2026-07-01 20:47:45.144763637 +0000 UTC m=+49.753206512" Jul 01 20:47:45.551311 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.551217 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85c9c87d46-qzd6s"] Jul 01 20:47:45.568964 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.568936 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85c9c87d46-qzd6s"] Jul 01 20:47:45.569085 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.569054 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.572604 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.572276 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 01 20:47:45.572604 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.572291 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 01 20:47:45.572604 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.572337 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 01 20:47:45.572604 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.572354 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-gf74g\"" Jul 01 20:47:45.572604 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.572291 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 20:47:45.572604 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.572292 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-be0p9n9qu5jnu\"" Jul 01 20:47:45.674072 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.674044 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-secret-metrics-server-tls\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.674241 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.674134 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/999102f7-ea92-4cf4-a889-4cf989dbca4a-audit-log\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.674241 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.674190 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999102f7-ea92-4cf4-a889-4cf989dbca4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.674365 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.674269 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9bgk\" (UniqueName: \"kubernetes.io/projected/999102f7-ea92-4cf4-a889-4cf989dbca4a-kube-api-access-r9bgk\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.674365 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.674302 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/999102f7-ea92-4cf4-a889-4cf989dbca4a-metrics-server-audit-profiles\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.674365 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.674335 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-secret-metrics-server-client-certs\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.674365 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.674355 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-client-ca-bundle\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.719367 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.719339 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5694969698-8zwdm"] Jul 01 20:47:45.722823 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:47:45.722797 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64e2477_7f8f_4ed4_b0bd_51f6f2f31252.slice/crio-953bec8dcb35140f7a45a17efc2889938521523d3487193c361e9809fbedf58c WatchSource:0}: Error finding container 953bec8dcb35140f7a45a17efc2889938521523d3487193c361e9809fbedf58c: Status 404 returned error can't find the container with id 953bec8dcb35140f7a45a17efc2889938521523d3487193c361e9809fbedf58c Jul 01 20:47:45.775292 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.775240 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/999102f7-ea92-4cf4-a889-4cf989dbca4a-audit-log\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.775411 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.775322 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999102f7-ea92-4cf4-a889-4cf989dbca4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.775411 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.775354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9bgk\" (UniqueName: \"kubernetes.io/projected/999102f7-ea92-4cf4-a889-4cf989dbca4a-kube-api-access-r9bgk\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.775411 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.775371 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/999102f7-ea92-4cf4-a889-4cf989dbca4a-metrics-server-audit-profiles\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.775519 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.775411 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-secret-metrics-server-client-certs\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.775519 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.775435 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-client-ca-bundle\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.775519 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.775466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-secret-metrics-server-tls\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.775651 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.775598 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/999102f7-ea92-4cf4-a889-4cf989dbca4a-audit-log\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.776305 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.776087 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/999102f7-ea92-4cf4-a889-4cf989dbca4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.776305 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.776276 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/999102f7-ea92-4cf4-a889-4cf989dbca4a-metrics-server-audit-profiles\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.777889 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.777853 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-secret-metrics-server-tls\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.778096 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.778074 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-secret-metrics-server-client-certs\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.778139 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.778078 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/999102f7-ea92-4cf4-a889-4cf989dbca4a-client-ca-bundle\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.788848 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.788831 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9bgk\" (UniqueName: \"kubernetes.io/projected/999102f7-ea92-4cf4-a889-4cf989dbca4a-kube-api-access-r9bgk\") pod \"metrics-server-85c9c87d46-qzd6s\" (UID: \"999102f7-ea92-4cf4-a889-4cf989dbca4a\") " pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.882103 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.882073 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:47:45.984617 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:45.984583 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-t45xb"] Jul 01 20:47:46.010065 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.010023 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-t45xb"] Jul 01 20:47:46.010203 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.010153 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:46.012561 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.012540 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7mpqn\"" Jul 01 20:47:46.012844 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.012824 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 01 20:47:46.078298 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.078270 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2kbf\" (UniqueName: \"kubernetes.io/projected/531e256d-55e5-4517-a876-90925897cb2e-kube-api-access-g2kbf\") pod \"monitoring-plugin-cd8656b8-t45xb\" (UID: \"531e256d-55e5-4517-a876-90925897cb2e\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:46.078460 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.078321 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/531e256d-55e5-4517-a876-90925897cb2e-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-t45xb\" (UID: \"531e256d-55e5-4517-a876-90925897cb2e\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:46.128222 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.128145 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" event={"ID":"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252","Type":"ContainerStarted","Data":"953bec8dcb35140f7a45a17efc2889938521523d3487193c361e9809fbedf58c"} Jul 01 20:47:46.179472 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.179217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/531e256d-55e5-4517-a876-90925897cb2e-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-t45xb\" (UID: \"531e256d-55e5-4517-a876-90925897cb2e\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:46.179596 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.179536 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2kbf\" (UniqueName: \"kubernetes.io/projected/531e256d-55e5-4517-a876-90925897cb2e-kube-api-access-g2kbf\") pod \"monitoring-plugin-cd8656b8-t45xb\" (UID: \"531e256d-55e5-4517-a876-90925897cb2e\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:46.182299 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.182227 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/531e256d-55e5-4517-a876-90925897cb2e-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-t45xb\" (UID: \"531e256d-55e5-4517-a876-90925897cb2e\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:46.190427 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.190386 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2kbf\" (UniqueName: \"kubernetes.io/projected/531e256d-55e5-4517-a876-90925897cb2e-kube-api-access-g2kbf\") pod \"monitoring-plugin-cd8656b8-t45xb\" (UID: \"531e256d-55e5-4517-a876-90925897cb2e\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:46.243396 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.243360 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85c9c87d46-qzd6s"] Jul 01 20:47:46.247119 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:47:46.247094 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999102f7_ea92_4cf4_a889_4cf989dbca4a.slice/crio-c35cf20a94335acd9443d1c037cae02ec4d9a83b21c6b62e8d7853556db5573f WatchSource:0}: Error finding container c35cf20a94335acd9443d1c037cae02ec4d9a83b21c6b62e8d7853556db5573f: Status 404 returned error can't find the container with id c35cf20a94335acd9443d1c037cae02ec4d9a83b21c6b62e8d7853556db5573f Jul 01 20:47:46.320411 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.320344 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:46.512714 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:46.512676 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-t45xb"] Jul 01 20:47:46.517776 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:47:46.517742 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531e256d_55e5_4517_a876_90925897cb2e.slice/crio-a3bd00b642044d8cdb1c8e307219c32c1e779f04300b9cb3b1824ff4cdc9ab08 WatchSource:0}: Error finding container a3bd00b642044d8cdb1c8e307219c32c1e779f04300b9cb3b1824ff4cdc9ab08: Status 404 returned error can't find the container with id a3bd00b642044d8cdb1c8e307219c32c1e779f04300b9cb3b1824ff4cdc9ab08 Jul 01 20:47:47.134496 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:47.134457 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dcd6fc8c-9vsqd" event={"ID":"42aaee9d-0478-4478-9a7b-afe423df82f6","Type":"ContainerStarted","Data":"8f058c224208f360e87afff07a674438f7531420e2dd8687972e35670c69b362"} Jul 01 20:47:47.136820 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:47.136768 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" event={"ID":"531e256d-55e5-4517-a876-90925897cb2e","Type":"ContainerStarted","Data":"a3bd00b642044d8cdb1c8e307219c32c1e779f04300b9cb3b1824ff4cdc9ab08"} Jul 01 20:47:47.138419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:47.138396 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" event={"ID":"999102f7-ea92-4cf4-a889-4cf989dbca4a","Type":"ContainerStarted","Data":"c35cf20a94335acd9443d1c037cae02ec4d9a83b21c6b62e8d7853556db5573f"} Jul 01 20:47:47.157133 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:47.157090 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64dcd6fc8c-9vsqd" podStartSLOduration=2.520961237 podStartE2EDuration="6.157076001s" totalImagesPullingTime="3.636114764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:47.15536468 +0000 UTC m=+51.763807556" watchObservedRunningTime="2026-07-01 20:47:47.157076001 +0000 UTC m=+51.765518875" Jul 01 20:47:51.154918 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.154886 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" event={"ID":"531e256d-55e5-4517-a876-90925897cb2e","Type":"ContainerStarted","Data":"e0185672141e4fb833ae0668d33ec91de8d72eacada5b14494d4a2d4fe256518"} Jul 01 20:47:51.155365 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.155086 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:51.159436 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.159395 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" event={"ID":"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252","Type":"ContainerStarted","Data":"bbcd8e3f1f129e78062345159a1bc14b5768aa274e2dd7d4a7ce4835baada35a"} Jul 01 20:47:51.159547 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.159475 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" event={"ID":"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252","Type":"ContainerStarted","Data":"836e7afce243943db1aaca8fb1dbdbef75ed4d67d220ed9d487b81365045d57b"} Jul 01 20:47:51.159547 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.159522 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" event={"ID":"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252","Type":"ContainerStarted","Data":"4c66e13952bfb9148bc11f89bee14de9728e6d3e6e87e01630da7c919dd9d789"} Jul 01 20:47:51.159640 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.159563 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" event={"ID":"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252","Type":"ContainerStarted","Data":"9983d9cef1a4f835483dfec7ac69f7f928c509a3b1c8ccc4ad56d2f4748d5765"} Jul 01 20:47:51.159640 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.159604 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" event={"ID":"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252","Type":"ContainerStarted","Data":"ec32bd0c14623e046890763ac4b22e0ec0bfceed08e08cf74db207f568ba45f2"} Jul 01 20:47:51.161556 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.161531 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" Jul 01 20:47:51.162303 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.162278 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" event={"ID":"999102f7-ea92-4cf4-a889-4cf989dbca4a","Type":"ContainerStarted","Data":"08498d19c9b3f8f9c2860645f68ed0b763402609d57b944cf983a1db661c0bd0"} Jul 01 20:47:51.164569 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.164545 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dc864" event={"ID":"e5117067-8f06-466b-aa57-be171670744e","Type":"ContainerStarted","Data":"e9081ea1032ebf6389a48846aad7d66f68971674158bd360b831e68000df6dd4"} Jul 01 20:47:51.171637 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.171594 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-t45xb" podStartSLOduration=2.498949446 podStartE2EDuration="6.171580801s" totalImagesPullingTime="3.672631355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:51.170608936 +0000 UTC m=+55.779051811" watchObservedRunningTime="2026-07-01 20:47:51.171580801 +0000 UTC m=+55.780023673" Jul 01 20:47:51.188505 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.188449 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" podStartSLOduration=2.276752756 podStartE2EDuration="6.188430617s" totalImagesPullingTime="3.911677861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:51.187118027 +0000 UTC m=+55.795560903" watchObservedRunningTime="2026-07-01 20:47:51.188430617 +0000 UTC m=+55.796873496" Jul 01 20:47:51.205993 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:51.205945 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dc864" podStartSLOduration=2.311174545 podStartE2EDuration="16.205931477s" totalImagesPullingTime="13.894756932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:47:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:51.20520237 +0000 UTC m=+55.813645245" watchObservedRunningTime="2026-07-01 20:47:51.205931477 +0000 UTC m=+55.814374354" Jul 01 20:47:52.106673 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:52.106640 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:52.106853 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:52.106715 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:52.111602 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:52.111577 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:52.170577 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:52.170550 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" event={"ID":"a64e2477-7f8f-4ed4-b0bd-51f6f2f31252","Type":"ContainerStarted","Data":"98da77c838836db1c5d3bd7b5f27281d333ac479e3ea1a4f4f62dcaedda20b68"} Jul 01 20:47:52.174744 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:52.174722 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:47:52.202229 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:52.202185 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" podStartSLOduration=3.12767625 podStartE2EDuration="8.202172351s" totalImagesPullingTime="5.074496101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:47:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:47:52.201106144 +0000 UTC m=+56.809549019" watchObservedRunningTime="2026-07-01 20:47:52.202172351 +0000 UTC m=+56.810615225" Jul 01 20:47:53.174179 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:53.174145 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:47:54.125326 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:54.125293 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-85h8x" Jul 01 20:47:55.034829 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:55.034796 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6r267" Jul 01 20:47:59.089129 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:59.089098 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7996c789b7-qvjxp" Jul 01 20:47:59.183664 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:47:59.183638 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5694969698-8zwdm" Jul 01 20:48:01.611481 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.611433 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:48:01.614988 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.614969 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 20:48:01.624308 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.624286 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/220e4077-2a30-4706-863a-6ec30801b3f3-metrics-certs\") pod \"network-metrics-daemon-kkfpv\" (UID: \"220e4077-2a30-4706-863a-6ec30801b3f3\") " pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:48:01.654285 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.654243 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7lgrj\"" Jul 01 20:48:01.662402 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.662383 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkfpv" Jul 01 20:48:01.712162 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.712137 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:48:01.712297 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.712180 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:48:01.716098 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.716078 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 20:48:01.716211 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.716129 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 20:48:01.725287 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.725252 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1172c283-88c3-407e-8c99-4af696b5984c-original-pull-secret\") pod \"global-pull-secret-syncer-2ntmn\" (UID: \"1172c283-88c3-407e-8c99-4af696b5984c\") " pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:48:01.725416 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.725399 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 20:48:01.736031 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.736004 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-846pb\" (UniqueName: \"kubernetes.io/projected/af091fbd-26ff-4648-9042-fa91ba493331-kube-api-access-846pb\") pod \"network-check-target-xj5kn\" (UID: \"af091fbd-26ff-4648-9042-fa91ba493331\") " pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:48:01.752574 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.752537 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2ntmn" Jul 01 20:48:01.764561 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.764531 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r694q\"" Jul 01 20:48:01.772273 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.772237 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:48:01.858608 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.858583 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kkfpv"] Jul 01 20:48:01.860902 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:48:01.860876 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220e4077_2a30_4706_863a_6ec30801b3f3.slice/crio-0110cc380de300c7ebe567de026f03ba665732ed30e00c5197b0ecfe7a89aae2 WatchSource:0}: Error finding container 0110cc380de300c7ebe567de026f03ba665732ed30e00c5197b0ecfe7a89aae2: Status 404 returned error can't find the container with id 0110cc380de300c7ebe567de026f03ba665732ed30e00c5197b0ecfe7a89aae2 Jul 01 20:48:01.951457 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.951427 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2ntmn"] Jul 01 20:48:01.954163 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:48:01.954134 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1172c283_88c3_407e_8c99_4af696b5984c.slice/crio-415db615ae5dd51652cdca0659b3a2560a2d1f594bffb69dd41cae1fc989cd4c WatchSource:0}: Error finding container 415db615ae5dd51652cdca0659b3a2560a2d1f594bffb69dd41cae1fc989cd4c: Status 404 returned error can't find the container with id 415db615ae5dd51652cdca0659b3a2560a2d1f594bffb69dd41cae1fc989cd4c Jul 01 20:48:01.981086 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:01.981054 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xj5kn"] Jul 01 20:48:01.987232 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:48:01.987205 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf091fbd_26ff_4648_9042_fa91ba493331.slice/crio-f90c585ff9c5713e85d1f2d1cf7d4a0f6b98825a686863555f0c14d647f3ae4b WatchSource:0}: Error finding container f90c585ff9c5713e85d1f2d1cf7d4a0f6b98825a686863555f0c14d647f3ae4b: Status 404 returned error can't find the container with id f90c585ff9c5713e85d1f2d1cf7d4a0f6b98825a686863555f0c14d647f3ae4b Jul 01 20:48:02.204542 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:02.204457 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xj5kn" event={"ID":"af091fbd-26ff-4648-9042-fa91ba493331","Type":"ContainerStarted","Data":"f90c585ff9c5713e85d1f2d1cf7d4a0f6b98825a686863555f0c14d647f3ae4b"} Jul 01 20:48:02.205608 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:02.205585 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2ntmn" event={"ID":"1172c283-88c3-407e-8c99-4af696b5984c","Type":"ContainerStarted","Data":"415db615ae5dd51652cdca0659b3a2560a2d1f594bffb69dd41cae1fc989cd4c"} Jul 01 20:48:02.206627 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:02.206603 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kkfpv" event={"ID":"220e4077-2a30-4706-863a-6ec30801b3f3","Type":"ContainerStarted","Data":"0110cc380de300c7ebe567de026f03ba665732ed30e00c5197b0ecfe7a89aae2"} Jul 01 20:48:03.218406 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:03.218361 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kkfpv" event={"ID":"220e4077-2a30-4706-863a-6ec30801b3f3","Type":"ContainerStarted","Data":"2623c11ec74dc4fbc9deb4dbea46349c324735b31244eda08c7696333535bb15"} Jul 01 20:48:04.226314 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:04.226249 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kkfpv" event={"ID":"220e4077-2a30-4706-863a-6ec30801b3f3","Type":"ContainerStarted","Data":"9a6aaa1daedb1169b764759f4d2fc3decf79cd0c96ebcea3be3204f7c0694be9"} Jul 01 20:48:04.254745 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:04.254684 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kkfpv" podStartSLOduration=68.08200875 podStartE2EDuration="1m9.254668188s" totalImagesPullingTime="1.172659438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:48:04.252371341 +0000 UTC m=+68.860814221" watchObservedRunningTime="2026-07-01 20:48:04.254668188 +0000 UTC m=+68.863111066" Jul 01 20:48:05.882649 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:05.882614 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:48:05.883180 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:05.882681 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:48:06.164841 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:06.164743 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64dcd6fc8c-9vsqd"] Jul 01 20:48:07.239354 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:07.239310 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xj5kn" event={"ID":"af091fbd-26ff-4648-9042-fa91ba493331","Type":"ContainerStarted","Data":"b2d1a59a9e4f78831ded2512dcd38a89599e50dc2d9b0d303ce79664feb9bffd"} Jul 01 20:48:07.239794 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:07.239441 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:48:07.240744 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:07.240722 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2ntmn" event={"ID":"1172c283-88c3-407e-8c99-4af696b5984c","Type":"ContainerStarted","Data":"2128c2d3cda2422b6a4b03634d2b3616bd9731d19416ee68855cde6d21e8bd1e"} Jul 01 20:48:07.255452 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:07.255382 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xj5kn" podStartSLOduration=66.65797287 podStartE2EDuration="1m11.255365208s" totalImagesPullingTime="4.597392338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:48:07.254783897 +0000 UTC m=+71.863226783" watchObservedRunningTime="2026-07-01 20:48:07.255365208 +0000 UTC m=+71.863808085" Jul 01 20:48:07.269547 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:07.269496 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2ntmn" podStartSLOduration=66.636155056 podStartE2EDuration="1m11.269480477s" totalImagesPullingTime="4.633325421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:48:07.269154546 +0000 UTC m=+71.877597421" watchObservedRunningTime="2026-07-01 20:48:07.269480477 +0000 UTC m=+71.877923353" Jul 01 20:48:25.887609 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:25.887487 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:48:25.891388 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:25.891369 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85c9c87d46-qzd6s" Jul 01 20:48:31.188155 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.188114 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-64dcd6fc8c-9vsqd" podUID="42aaee9d-0478-4478-9a7b-afe423df82f6" containerName="console" containerID="cri-o://8f058c224208f360e87afff07a674438f7531420e2dd8687972e35670c69b362" gracePeriod=15 Jul 01 20:48:31.329719 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.329686 2558 generic.go:356] "Generic (PLEG): container finished" podID="42aaee9d-0478-4478-9a7b-afe423df82f6" containerID="8f058c224208f360e87afff07a674438f7531420e2dd8687972e35670c69b362" exitCode=0 Jul 01 20:48:31.329873 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.329768 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dcd6fc8c-9vsqd" event={"ID":"42aaee9d-0478-4478-9a7b-afe423df82f6","Type":"ContainerDied","Data":"8f058c224208f360e87afff07a674438f7531420e2dd8687972e35670c69b362"} Jul 01 20:48:31.433623 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.433600 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:48:31.442051 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.441983 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-oauth-config\") pod \"42aaee9d-0478-4478-9a7b-afe423df82f6\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " Jul 01 20:48:31.442051 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.442017 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-trusted-ca-bundle\") pod \"42aaee9d-0478-4478-9a7b-afe423df82f6\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " Jul 01 20:48:31.442051 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.442038 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-service-ca\") pod \"42aaee9d-0478-4478-9a7b-afe423df82f6\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " Jul 01 20:48:31.442051 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.442055 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-serving-cert\") pod \"42aaee9d-0478-4478-9a7b-afe423df82f6\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " Jul 01 20:48:31.442352 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.442170 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn2sx\" (UniqueName: \"kubernetes.io/projected/42aaee9d-0478-4478-9a7b-afe423df82f6-kube-api-access-rn2sx\") pod \"42aaee9d-0478-4478-9a7b-afe423df82f6\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " Jul 01 20:48:31.442352 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.442231 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-oauth-serving-cert\") pod \"42aaee9d-0478-4478-9a7b-afe423df82f6\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " Jul 01 20:48:31.442495 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.442466 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-service-ca" (OuterVolumeSpecName: "service-ca") pod "42aaee9d-0478-4478-9a7b-afe423df82f6" (UID: "42aaee9d-0478-4478-9a7b-afe423df82f6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:48:31.442545 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.442491 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "42aaee9d-0478-4478-9a7b-afe423df82f6" (UID: "42aaee9d-0478-4478-9a7b-afe423df82f6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:48:31.442816 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.442760 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "42aaee9d-0478-4478-9a7b-afe423df82f6" (UID: "42aaee9d-0478-4478-9a7b-afe423df82f6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:48:31.444565 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.444534 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42aaee9d-0478-4478-9a7b-afe423df82f6-kube-api-access-rn2sx" (OuterVolumeSpecName: "kube-api-access-rn2sx") pod "42aaee9d-0478-4478-9a7b-afe423df82f6" (UID: "42aaee9d-0478-4478-9a7b-afe423df82f6"). InnerVolumeSpecName "kube-api-access-rn2sx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 20:48:31.444663 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.444539 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "42aaee9d-0478-4478-9a7b-afe423df82f6" (UID: "42aaee9d-0478-4478-9a7b-afe423df82f6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 20:48:31.444663 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.444587 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "42aaee9d-0478-4478-9a7b-afe423df82f6" (UID: "42aaee9d-0478-4478-9a7b-afe423df82f6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 20:48:31.543090 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.543044 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-console-config\") pod \"42aaee9d-0478-4478-9a7b-afe423df82f6\" (UID: \"42aaee9d-0478-4478-9a7b-afe423df82f6\") " Jul 01 20:48:31.543312 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.543300 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn2sx\" (UniqueName: \"kubernetes.io/projected/42aaee9d-0478-4478-9a7b-afe423df82f6-kube-api-access-rn2sx\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:48:31.543390 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.543319 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-oauth-serving-cert\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:48:31.543390 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.543332 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-oauth-config\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:48:31.543390 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.543346 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-trusted-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:48:31.543390 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.543359 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-service-ca\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:48:31.543390 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.543372 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42aaee9d-0478-4478-9a7b-afe423df82f6-console-serving-cert\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:48:31.543545 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.543411 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-console-config" (OuterVolumeSpecName: "console-config") pod "42aaee9d-0478-4478-9a7b-afe423df82f6" (UID: "42aaee9d-0478-4478-9a7b-afe423df82f6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:48:31.644567 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:31.644533 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42aaee9d-0478-4478-9a7b-afe423df82f6-console-config\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:48:32.334091 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:32.334059 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dcd6fc8c-9vsqd" event={"ID":"42aaee9d-0478-4478-9a7b-afe423df82f6","Type":"ContainerDied","Data":"b58454d1dceed840bc818aedbd2b292547f86fe172509589d1b2f0debf996109"} Jul 01 20:48:32.334543 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:32.334116 2558 scope.go:117] "RemoveContainer" containerID="8f058c224208f360e87afff07a674438f7531420e2dd8687972e35670c69b362" Jul 01 20:48:32.334543 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:32.334066 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dcd6fc8c-9vsqd" Jul 01 20:48:32.351032 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:32.351009 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64dcd6fc8c-9vsqd"] Jul 01 20:48:32.354720 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:32.354700 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64dcd6fc8c-9vsqd"] Jul 01 20:48:33.843582 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:33.843534 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42aaee9d-0478-4478-9a7b-afe423df82f6" path="/var/lib/kubelet/pods/42aaee9d-0478-4478-9a7b-afe423df82f6/volumes" Jul 01 20:48:38.249321 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:48:38.249285 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xj5kn" Jul 01 20:49:03.713432 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.713393 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76cd7d5f7f-dq4ws"] Jul 01 20:49:03.713907 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.713652 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42aaee9d-0478-4478-9a7b-afe423df82f6" containerName="console" Jul 01 20:49:03.713907 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.713663 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="42aaee9d-0478-4478-9a7b-afe423df82f6" containerName="console" Jul 01 20:49:03.713907 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.713703 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="42aaee9d-0478-4478-9a7b-afe423df82f6" containerName="console" Jul 01 20:49:03.716497 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.716479 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.718804 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.718783 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 01 20:49:03.718928 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.718858 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 01 20:49:03.719081 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.719066 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 01 20:49:03.719666 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.719648 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 01 20:49:03.720027 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.719997 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 01 20:49:03.720101 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.720083 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-t872r\"" Jul 01 20:49:03.720155 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.720086 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 01 20:49:03.720208 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.720199 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 01 20:49:03.724694 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.724677 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 01 20:49:03.731013 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.730015 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cd7d5f7f-dq4ws"] Jul 01 20:49:03.772362 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.772316 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-oauth-config\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.772362 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.772368 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw2zw\" (UniqueName: \"kubernetes.io/projected/e19ea2ca-4447-4515-ba34-480fa85ea699-kube-api-access-nw2zw\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.772362 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.772390 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-oauth-serving-cert\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.772683 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.772455 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-console-config\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.772683 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.772541 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-trusted-ca-bundle\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.772683 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.772580 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-service-ca\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.772683 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.772631 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-serving-cert\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.873700 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.873652 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-trusted-ca-bundle\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.873700 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.873704 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-service-ca\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.873948 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.873737 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-serving-cert\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.873948 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.873768 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-oauth-config\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.873948 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.873787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw2zw\" (UniqueName: \"kubernetes.io/projected/e19ea2ca-4447-4515-ba34-480fa85ea699-kube-api-access-nw2zw\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.873948 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.873802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-oauth-serving-cert\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.873948 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.873830 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-console-config\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.874664 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.874596 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-oauth-serving-cert\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.874664 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.874597 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-console-config\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.874664 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.874597 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-service-ca\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.874943 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.874841 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-trusted-ca-bundle\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.876530 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.876497 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-oauth-config\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.876619 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.876551 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-serving-cert\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:03.882658 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:03.882637 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw2zw\" (UniqueName: \"kubernetes.io/projected/e19ea2ca-4447-4515-ba34-480fa85ea699-kube-api-access-nw2zw\") pod \"console-76cd7d5f7f-dq4ws\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:04.026415 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:04.026325 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:04.158464 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:04.158307 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cd7d5f7f-dq4ws"] Jul 01 20:49:04.161046 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:49:04.161019 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19ea2ca_4447_4515_ba34_480fa85ea699.slice/crio-92a037b030ddcaea6a38f6a6aedff7e3cfd1d0916b0e480aac5b9ea73a08250f WatchSource:0}: Error finding container 92a037b030ddcaea6a38f6a6aedff7e3cfd1d0916b0e480aac5b9ea73a08250f: Status 404 returned error can't find the container with id 92a037b030ddcaea6a38f6a6aedff7e3cfd1d0916b0e480aac5b9ea73a08250f Jul 01 20:49:04.445803 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:04.445759 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cd7d5f7f-dq4ws" event={"ID":"e19ea2ca-4447-4515-ba34-480fa85ea699","Type":"ContainerStarted","Data":"1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6"} Jul 01 20:49:04.446066 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:04.445815 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cd7d5f7f-dq4ws" event={"ID":"e19ea2ca-4447-4515-ba34-480fa85ea699","Type":"ContainerStarted","Data":"92a037b030ddcaea6a38f6a6aedff7e3cfd1d0916b0e480aac5b9ea73a08250f"} Jul 01 20:49:04.480547 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:04.480491 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76cd7d5f7f-dq4ws" podStartSLOduration=1.48047621 podStartE2EDuration="1.48047621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:49:04.478312673 +0000 UTC m=+129.086755552" watchObservedRunningTime="2026-07-01 20:49:04.48047621 +0000 UTC m=+129.088919084" Jul 01 20:49:14.027294 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:14.027241 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:14.027757 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:14.027337 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:14.032195 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:14.032175 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:14.484034 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:14.484006 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:49:53.967541 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:53.967464 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9"] Jul 01 20:49:53.969614 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:53.969599 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:53.971892 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:53.971873 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 01 20:49:53.971996 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:53.971950 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 01 20:49:53.972675 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:53.972661 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v8xfz\"" Jul 01 20:49:53.979281 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:53.979241 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9"] Jul 01 20:49:54.056965 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.056931 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.057139 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.056982 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdbh6\" (UniqueName: \"kubernetes.io/projected/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-kube-api-access-gdbh6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.057139 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.057051 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.157451 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.157413 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdbh6\" (UniqueName: \"kubernetes.io/projected/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-kube-api-access-gdbh6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.157606 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.157467 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.157606 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.157496 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.157802 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.157788 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.157851 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.157832 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.166163 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.166132 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdbh6\" (UniqueName: \"kubernetes.io/projected/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-kube-api-access-gdbh6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.279498 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.279407 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:49:54.406442 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.406407 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9"] Jul 01 20:49:54.409391 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:49:54.409362 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e0999fa_bc1f_4d18_9114_bd2e5752d4de.slice/crio-5126b3338396a748424efd51da3cd96f83c01869701c9d088f4a23c267bde7e7 WatchSource:0}: Error finding container 5126b3338396a748424efd51da3cd96f83c01869701c9d088f4a23c267bde7e7: Status 404 returned error can't find the container with id 5126b3338396a748424efd51da3cd96f83c01869701c9d088f4a23c267bde7e7 Jul 01 20:49:54.612461 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:49:54.612419 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" event={"ID":"2e0999fa-bc1f-4d18-9114-bd2e5752d4de","Type":"ContainerStarted","Data":"5126b3338396a748424efd51da3cd96f83c01869701c9d088f4a23c267bde7e7"} Jul 01 20:50:02.644482 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:02.644439 2558 generic.go:356] "Generic (PLEG): container finished" podID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerID="98ec5c800bbb58ab9c71137751f1d9024dd6ce7f4e75b961daf243bf74e6d2d4" exitCode=0 Jul 01 20:50:02.644860 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:02.644526 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" event={"ID":"2e0999fa-bc1f-4d18-9114-bd2e5752d4de","Type":"ContainerDied","Data":"98ec5c800bbb58ab9c71137751f1d9024dd6ce7f4e75b961daf243bf74e6d2d4"} Jul 01 20:50:06.660657 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:06.660620 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" event={"ID":"2e0999fa-bc1f-4d18-9114-bd2e5752d4de","Type":"ContainerStarted","Data":"f5dc8a8eca20b5e08d1d3fa754f58b10df880adebbf60ad6ce3443e8775d826c"} Jul 01 20:50:07.666147 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:07.666113 2558 generic.go:356] "Generic (PLEG): container finished" podID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerID="f5dc8a8eca20b5e08d1d3fa754f58b10df880adebbf60ad6ce3443e8775d826c" exitCode=0 Jul 01 20:50:07.666560 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:07.666179 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" event={"ID":"2e0999fa-bc1f-4d18-9114-bd2e5752d4de","Type":"ContainerDied","Data":"f5dc8a8eca20b5e08d1d3fa754f58b10df880adebbf60ad6ce3443e8775d826c"} Jul 01 20:50:16.704023 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:16.703986 2558 generic.go:356] "Generic (PLEG): container finished" podID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerID="c6800955743db63f84b4411eec11e5988f6b82b30b88e643271ddd5614ad2308" exitCode=0 Jul 01 20:50:16.704414 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:16.704036 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" event={"ID":"2e0999fa-bc1f-4d18-9114-bd2e5752d4de","Type":"ContainerDied","Data":"c6800955743db63f84b4411eec11e5988f6b82b30b88e643271ddd5614ad2308"} Jul 01 20:50:17.889952 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:17.889919 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:50:17.935998 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:17.935970 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-bundle\") pod \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " Jul 01 20:50:17.936171 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:17.936020 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdbh6\" (UniqueName: \"kubernetes.io/projected/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-kube-api-access-gdbh6\") pod \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " Jul 01 20:50:17.936171 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:17.936043 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-util\") pod \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\" (UID: \"2e0999fa-bc1f-4d18-9114-bd2e5752d4de\") " Jul 01 20:50:17.936637 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:17.936609 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-bundle" (OuterVolumeSpecName: "bundle") pod "2e0999fa-bc1f-4d18-9114-bd2e5752d4de" (UID: "2e0999fa-bc1f-4d18-9114-bd2e5752d4de"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 20:50:17.938403 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:17.938374 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-kube-api-access-gdbh6" (OuterVolumeSpecName: "kube-api-access-gdbh6") pod "2e0999fa-bc1f-4d18-9114-bd2e5752d4de" (UID: "2e0999fa-bc1f-4d18-9114-bd2e5752d4de"). InnerVolumeSpecName "kube-api-access-gdbh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 20:50:17.941373 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:17.941354 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-util" (OuterVolumeSpecName: "util") pod "2e0999fa-bc1f-4d18-9114-bd2e5752d4de" (UID: "2e0999fa-bc1f-4d18-9114-bd2e5752d4de"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 20:50:18.037443 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:18.037359 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:50:18.037443 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:18.037398 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdbh6\" (UniqueName: \"kubernetes.io/projected/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-kube-api-access-gdbh6\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:50:18.037443 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:18.037425 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0999fa-bc1f-4d18-9114-bd2e5752d4de-util\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:50:18.717659 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:18.717617 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" Jul 01 20:50:18.717659 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:18.717618 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589fdgf9" event={"ID":"2e0999fa-bc1f-4d18-9114-bd2e5752d4de","Type":"ContainerDied","Data":"5126b3338396a748424efd51da3cd96f83c01869701c9d088f4a23c267bde7e7"} Jul 01 20:50:18.717862 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:18.717750 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5126b3338396a748424efd51da3cd96f83c01869701c9d088f4a23c267bde7e7" Jul 01 20:50:21.295017 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.294954 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2"] Jul 01 20:50:21.295419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.295233 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerName="util" Jul 01 20:50:21.295419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.295243 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerName="util" Jul 01 20:50:21.295419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.295251 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerName="pull" Jul 01 20:50:21.295419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.295270 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerName="pull" Jul 01 20:50:21.295419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.295288 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerName="extract" Jul 01 20:50:21.295419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.295294 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerName="extract" Jul 01 20:50:21.295419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.295336 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e0999fa-bc1f-4d18-9114-bd2e5752d4de" containerName="extract" Jul 01 20:50:21.297217 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.297201 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:21.299478 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.299450 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 01 20:50:21.299734 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.299715 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-pw2qd\"" Jul 01 20:50:21.299932 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.299913 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 01 20:50:21.300036 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.300021 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 01 20:50:21.312557 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.312533 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2"] Jul 01 20:50:21.362769 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.362717 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrqtr\" (UniqueName: \"kubernetes.io/projected/fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c-kube-api-access-lrqtr\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2\" (UID: \"fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:21.362953 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.362789 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2\" (UID: \"fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:21.464068 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.464017 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrqtr\" (UniqueName: \"kubernetes.io/projected/fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c-kube-api-access-lrqtr\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2\" (UID: \"fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:21.464179 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.464095 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2\" (UID: \"fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:21.466503 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.466476 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2\" (UID: \"fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:21.473769 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.473749 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrqtr\" (UniqueName: \"kubernetes.io/projected/fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c-kube-api-access-lrqtr\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2\" (UID: \"fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:21.607634 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.607598 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:21.741571 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:21.741489 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2"] Jul 01 20:50:21.745432 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:50:21.745401 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0137e9_c1ef_43c1_92bd_3bfbabcbbd9c.slice/crio-935c2e7cf8ec2a916325d6d4c70dea4ca0ca15968bf3042cb8d1288c11502cfe WatchSource:0}: Error finding container 935c2e7cf8ec2a916325d6d4c70dea4ca0ca15968bf3042cb8d1288c11502cfe: Status 404 returned error can't find the container with id 935c2e7cf8ec2a916325d6d4c70dea4ca0ca15968bf3042cb8d1288c11502cfe Jul 01 20:50:22.738417 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:22.738376 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" event={"ID":"fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c","Type":"ContainerStarted","Data":"935c2e7cf8ec2a916325d6d4c70dea4ca0ca15968bf3042cb8d1288c11502cfe"} Jul 01 20:50:25.750843 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:25.750753 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" event={"ID":"fd0137e9-c1ef-43c1-92bd-3bfbabcbbd9c","Type":"ContainerStarted","Data":"a6f6e33e8466993c505fae7360c8affd3dc8e8f46eed3529357411173cb7bfcc"} Jul 01 20:50:25.751338 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:25.750892 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:25.776888 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:25.776834 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" podStartSLOduration=1.043097104 podStartE2EDuration="4.776816018s" totalImagesPullingTime="3.733718914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:50:25.775006918 +0000 UTC m=+210.383449793" watchObservedRunningTime="2026-07-01 20:50:25.776816018 +0000 UTC m=+210.385258895" Jul 01 20:50:26.207184 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.207152 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-hfqtv"] Jul 01 20:50:26.209215 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.209199 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.211163 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.211137 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 01 20:50:26.211309 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.211183 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 01 20:50:26.211309 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.211196 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-kprw7\"" Jul 01 20:50:26.219149 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.219114 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-hfqtv"] Jul 01 20:50:26.303150 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.303114 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/0ed907d9-c118-4525-a452-c43cb5df13a6-cabundle0\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.303343 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.303158 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pt49\" (UniqueName: \"kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-kube-api-access-2pt49\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.303343 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.303196 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-certificates\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.404355 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.404322 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/0ed907d9-c118-4525-a452-c43cb5df13a6-cabundle0\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.404355 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.404357 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pt49\" (UniqueName: \"kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-kube-api-access-2pt49\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.404558 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.404386 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-certificates\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.404558 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:50:26.404500 2558 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Jul 01 20:50:26.404558 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:50:26.404515 2558 secret.go:281] references non-existent secret key: ca.crt Jul 01 20:50:26.404558 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:50:26.404522 2558 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 01 20:50:26.404558 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:50:26.404533 2558 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-hfqtv: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 01 20:50:26.404737 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:50:26.404588 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-certificates podName:0ed907d9-c118-4525-a452-c43cb5df13a6 nodeName:}" failed. No retries permitted until 2026-07-01 20:50:26.904569665 +0000 UTC m=+211.513012542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-certificates") pod "keda-operator-68bff59c-hfqtv" (UID: "0ed907d9-c118-4525-a452-c43cb5df13a6") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 01 20:50:26.405099 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.405074 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/0ed907d9-c118-4525-a452-c43cb5df13a6-cabundle0\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.413245 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.413222 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pt49\" (UniqueName: \"kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-kube-api-access-2pt49\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.663295 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.663245 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw"] Jul 01 20:50:26.665479 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.665461 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.667852 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.667837 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 01 20:50:26.677707 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.677685 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw"] Jul 01 20:50:26.706799 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.706765 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/dbfdd79b-c670-48e2-b802-5d6730d1a736-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.706905 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.706804 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88rrb\" (UniqueName: \"kubernetes.io/projected/dbfdd79b-c670-48e2-b802-5d6730d1a736-kube-api-access-88rrb\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.706905 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.706886 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/dbfdd79b-c670-48e2-b802-5d6730d1a736-certificates\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.807698 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.807660 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/dbfdd79b-c670-48e2-b802-5d6730d1a736-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.808101 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.807703 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88rrb\" (UniqueName: \"kubernetes.io/projected/dbfdd79b-c670-48e2-b802-5d6730d1a736-kube-api-access-88rrb\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.808101 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.807743 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/dbfdd79b-c670-48e2-b802-5d6730d1a736-certificates\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.808101 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.808027 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/dbfdd79b-c670-48e2-b802-5d6730d1a736-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.810304 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.810286 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/dbfdd79b-c670-48e2-b802-5d6730d1a736-certificates\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.820372 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.820353 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88rrb\" (UniqueName: \"kubernetes.io/projected/dbfdd79b-c670-48e2-b802-5d6730d1a736-kube-api-access-88rrb\") pod \"keda-metrics-apiserver-559f75f947-qvwdw\" (UID: \"dbfdd79b-c670-48e2-b802-5d6730d1a736\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:26.908226 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.908196 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-certificates\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.910573 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.910553 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/0ed907d9-c118-4525-a452-c43cb5df13a6-certificates\") pod \"keda-operator-68bff59c-hfqtv\" (UID: \"0ed907d9-c118-4525-a452-c43cb5df13a6\") " pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:26.957715 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.957655 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-2rqh8"] Jul 01 20:50:26.960137 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.960124 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:26.962238 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.962216 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 01 20:50:26.969743 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.969722 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-2rqh8"] Jul 01 20:50:26.974513 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:26.974497 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:27.008978 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.008934 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9s4\" (UniqueName: \"kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-kube-api-access-4t9s4\") pod \"keda-admission-5fbd5c4644-2rqh8\" (UID: \"079d3ec6-8d45-47d4-88f9-602d2b9c0f66\") " pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:27.008978 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.008971 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-certificates\") pod \"keda-admission-5fbd5c4644-2rqh8\" (UID: \"079d3ec6-8d45-47d4-88f9-602d2b9c0f66\") " pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:27.106794 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.106769 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw"] Jul 01 20:50:27.109228 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:50:27.109197 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbfdd79b_c670_48e2_b802_5d6730d1a736.slice/crio-365d796f2e7994a711773e65393b8ee70cba0ca44043d2132d3b6745e5090c30 WatchSource:0}: Error finding container 365d796f2e7994a711773e65393b8ee70cba0ca44043d2132d3b6745e5090c30: Status 404 returned error can't find the container with id 365d796f2e7994a711773e65393b8ee70cba0ca44043d2132d3b6745e5090c30 Jul 01 20:50:27.109525 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.109505 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9s4\" (UniqueName: \"kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-kube-api-access-4t9s4\") pod \"keda-admission-5fbd5c4644-2rqh8\" (UID: \"079d3ec6-8d45-47d4-88f9-602d2b9c0f66\") " pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:27.109593 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.109540 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-certificates\") pod \"keda-admission-5fbd5c4644-2rqh8\" (UID: \"079d3ec6-8d45-47d4-88f9-602d2b9c0f66\") " pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:27.109701 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:50:27.109687 2558 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found Jul 01 20:50:27.109757 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:50:27.109712 2558 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-5fbd5c4644-2rqh8: secret "keda-admission-webhooks-certs" not found Jul 01 20:50:27.109804 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:50:27.109776 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-certificates podName:079d3ec6-8d45-47d4-88f9-602d2b9c0f66 nodeName:}" failed. No retries permitted until 2026-07-01 20:50:27.609755465 +0000 UTC m=+212.218198338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-certificates") pod "keda-admission-5fbd5c4644-2rqh8" (UID: "079d3ec6-8d45-47d4-88f9-602d2b9c0f66") : secret "keda-admission-webhooks-certs" not found Jul 01 20:50:27.118273 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.118231 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9s4\" (UniqueName: \"kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-kube-api-access-4t9s4\") pod \"keda-admission-5fbd5c4644-2rqh8\" (UID: \"079d3ec6-8d45-47d4-88f9-602d2b9c0f66\") " pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:27.120587 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.120566 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:27.248012 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.247990 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-hfqtv"] Jul 01 20:50:27.250484 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:50:27.250458 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed907d9_c118_4525_a452_c43cb5df13a6.slice/crio-9e7d4c6dea91ed125532f1a92e1913e37a72e75c32fd4085eb64c6efc7d6ac31 WatchSource:0}: Error finding container 9e7d4c6dea91ed125532f1a92e1913e37a72e75c32fd4085eb64c6efc7d6ac31: Status 404 returned error can't find the container with id 9e7d4c6dea91ed125532f1a92e1913e37a72e75c32fd4085eb64c6efc7d6ac31 Jul 01 20:50:27.614981 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.614948 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-certificates\") pod \"keda-admission-5fbd5c4644-2rqh8\" (UID: \"079d3ec6-8d45-47d4-88f9-602d2b9c0f66\") " pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:27.617524 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.617506 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/079d3ec6-8d45-47d4-88f9-602d2b9c0f66-certificates\") pod \"keda-admission-5fbd5c4644-2rqh8\" (UID: \"079d3ec6-8d45-47d4-88f9-602d2b9c0f66\") " pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:27.758920 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.758881 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" event={"ID":"dbfdd79b-c670-48e2-b802-5d6730d1a736","Type":"ContainerStarted","Data":"365d796f2e7994a711773e65393b8ee70cba0ca44043d2132d3b6745e5090c30"} Jul 01 20:50:27.759832 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.759812 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-hfqtv" event={"ID":"0ed907d9-c118-4525-a452-c43cb5df13a6","Type":"ContainerStarted","Data":"9e7d4c6dea91ed125532f1a92e1913e37a72e75c32fd4085eb64c6efc7d6ac31"} Jul 01 20:50:27.870123 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:27.870048 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:28.008670 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:28.008635 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-2rqh8"] Jul 01 20:50:28.011886 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:50:28.011859 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079d3ec6_8d45_47d4_88f9_602d2b9c0f66.slice/crio-e896c310b0cb2d704e83c168a101c60b3e4d74f4b973e603069f6eda75158b3f WatchSource:0}: Error finding container e896c310b0cb2d704e83c168a101c60b3e4d74f4b973e603069f6eda75158b3f: Status 404 returned error can't find the container with id e896c310b0cb2d704e83c168a101c60b3e4d74f4b973e603069f6eda75158b3f Jul 01 20:50:28.765916 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:28.765868 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" event={"ID":"079d3ec6-8d45-47d4-88f9-602d2b9c0f66","Type":"ContainerStarted","Data":"e896c310b0cb2d704e83c168a101c60b3e4d74f4b973e603069f6eda75158b3f"} Jul 01 20:50:31.781324 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.781291 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-hfqtv" event={"ID":"0ed907d9-c118-4525-a452-c43cb5df13a6","Type":"ContainerStarted","Data":"1b3bdb6a0e6bccf6ebca6d6413be9393ec334e4f7903e4bcea7f14e522505e7f"} Jul 01 20:50:31.781745 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.781429 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:31.783062 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.783033 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" event={"ID":"dbfdd79b-c670-48e2-b802-5d6730d1a736","Type":"ContainerStarted","Data":"e75567ef018050afde27a0adc939a8afd0ce9a9fe561324797f2bb9260fa5ffd"} Jul 01 20:50:31.783187 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.783127 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:31.784601 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.784576 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" event={"ID":"079d3ec6-8d45-47d4-88f9-602d2b9c0f66","Type":"ContainerStarted","Data":"53dbbc5560509fa333a5495e863e16cca9ba6ae632472a344f1735711149b3f6"} Jul 01 20:50:31.784727 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.784713 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:50:31.803141 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.803056 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-hfqtv" podStartSLOduration=1.448092997 podStartE2EDuration="5.80303433s" totalImagesPullingTime="4.354941333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:50:31.800178007 +0000 UTC m=+216.408620886" watchObservedRunningTime="2026-07-01 20:50:31.80303433 +0000 UTC m=+216.411477207" Jul 01 20:50:31.822891 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.822840 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" podStartSLOduration=1.323867296 podStartE2EDuration="5.822818303s" totalImagesPullingTime="4.498951007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:50:31.821182453 +0000 UTC m=+216.429625340" watchObservedRunningTime="2026-07-01 20:50:31.822818303 +0000 UTC m=+216.431261181" Jul 01 20:50:31.843335 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:31.843208 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" podStartSLOduration=2.24677275 podStartE2EDuration="5.843187381s" totalImagesPullingTime="3.596414631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:50:31.841128907 +0000 UTC m=+216.449571784" watchObservedRunningTime="2026-07-01 20:50:31.843187381 +0000 UTC m=+216.451630254" Jul 01 20:50:42.793234 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:42.793202 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qvwdw" Jul 01 20:50:46.756976 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:46.756947 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-6fzm2" Jul 01 20:50:52.791427 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:52.791390 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-hfqtv" Jul 01 20:50:52.791894 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:50:52.791768 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-2rqh8" Jul 01 20:51:33.656986 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.656915 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-67485586b-pqcfn"] Jul 01 20:51:33.659006 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.658990 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:33.660903 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.660880 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-hqzz6\"" Jul 01 20:51:33.661001 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.660945 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 01 20:51:33.661622 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.661607 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 01 20:51:33.661693 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.661619 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 01 20:51:33.663635 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.663617 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-7d672"] Jul 01 20:51:33.666178 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.666159 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:33.667974 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.667956 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 01 20:51:33.668069 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.667973 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-4cx8s\"" Jul 01 20:51:33.670048 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.670028 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-67485586b-pqcfn"] Jul 01 20:51:33.675908 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.675889 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-7d672"] Jul 01 20:51:33.734138 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.734114 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks7b6\" (UniqueName: \"kubernetes.io/projected/596568a3-e6ef-4595-8fdd-552ab2f99cc6-kube-api-access-ks7b6\") pod \"kserve-controller-manager-67485586b-pqcfn\" (UID: \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\") " pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:33.734281 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.734155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2mcf\" (UniqueName: \"kubernetes.io/projected/6d498d32-13ed-4938-ab98-77a4137dbe07-kube-api-access-v2mcf\") pod \"llmisvc-controller-manager-67d95fdc48-7d672\" (UID: \"6d498d32-13ed-4938-ab98-77a4137dbe07\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:33.734281 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.734178 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d498d32-13ed-4938-ab98-77a4137dbe07-cert\") pod \"llmisvc-controller-manager-67d95fdc48-7d672\" (UID: \"6d498d32-13ed-4938-ab98-77a4137dbe07\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:33.734362 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.734307 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/596568a3-e6ef-4595-8fdd-552ab2f99cc6-cert\") pod \"kserve-controller-manager-67485586b-pqcfn\" (UID: \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\") " pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:33.835521 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.835493 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/596568a3-e6ef-4595-8fdd-552ab2f99cc6-cert\") pod \"kserve-controller-manager-67485586b-pqcfn\" (UID: \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\") " pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:33.835614 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.835535 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks7b6\" (UniqueName: \"kubernetes.io/projected/596568a3-e6ef-4595-8fdd-552ab2f99cc6-kube-api-access-ks7b6\") pod \"kserve-controller-manager-67485586b-pqcfn\" (UID: \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\") " pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:33.835677 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.835656 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2mcf\" (UniqueName: \"kubernetes.io/projected/6d498d32-13ed-4938-ab98-77a4137dbe07-kube-api-access-v2mcf\") pod \"llmisvc-controller-manager-67d95fdc48-7d672\" (UID: \"6d498d32-13ed-4938-ab98-77a4137dbe07\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:33.835711 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.835700 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d498d32-13ed-4938-ab98-77a4137dbe07-cert\") pod \"llmisvc-controller-manager-67d95fdc48-7d672\" (UID: \"6d498d32-13ed-4938-ab98-77a4137dbe07\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:33.835814 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:51:33.835801 2558 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 01 20:51:33.835877 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:51:33.835868 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d498d32-13ed-4938-ab98-77a4137dbe07-cert podName:6d498d32-13ed-4938-ab98-77a4137dbe07 nodeName:}" failed. No retries permitted until 2026-07-01 20:51:34.335850308 +0000 UTC m=+278.944293170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6d498d32-13ed-4938-ab98-77a4137dbe07-cert") pod "llmisvc-controller-manager-67d95fdc48-7d672" (UID: "6d498d32-13ed-4938-ab98-77a4137dbe07") : secret "llmisvc-webhook-server-cert" not found Jul 01 20:51:33.838096 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.838062 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/596568a3-e6ef-4595-8fdd-552ab2f99cc6-cert\") pod \"kserve-controller-manager-67485586b-pqcfn\" (UID: \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\") " pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:33.846066 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.846041 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2mcf\" (UniqueName: \"kubernetes.io/projected/6d498d32-13ed-4938-ab98-77a4137dbe07-kube-api-access-v2mcf\") pod \"llmisvc-controller-manager-67d95fdc48-7d672\" (UID: \"6d498d32-13ed-4938-ab98-77a4137dbe07\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:33.846742 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.846726 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks7b6\" (UniqueName: \"kubernetes.io/projected/596568a3-e6ef-4595-8fdd-552ab2f99cc6-kube-api-access-ks7b6\") pod \"kserve-controller-manager-67485586b-pqcfn\" (UID: \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\") " pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:33.969400 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:33.969307 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:34.097837 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:34.097587 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-67485586b-pqcfn"] Jul 01 20:51:34.339553 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:34.339518 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d498d32-13ed-4938-ab98-77a4137dbe07-cert\") pod \"llmisvc-controller-manager-67d95fdc48-7d672\" (UID: \"6d498d32-13ed-4938-ab98-77a4137dbe07\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:34.341952 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:34.341925 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d498d32-13ed-4938-ab98-77a4137dbe07-cert\") pod \"llmisvc-controller-manager-67d95fdc48-7d672\" (UID: \"6d498d32-13ed-4938-ab98-77a4137dbe07\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:34.576723 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:34.576698 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:34.699217 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:34.699193 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-7d672"] Jul 01 20:51:34.701725 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:51:34.701682 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6d498d32_13ed_4938_ab98_77a4137dbe07.slice/crio-4d6776350718ee5e239ccaf258dd356a70eb62a9ffad5333f70ee5fd2f5dc827 WatchSource:0}: Error finding container 4d6776350718ee5e239ccaf258dd356a70eb62a9ffad5333f70ee5fd2f5dc827: Status 404 returned error can't find the container with id 4d6776350718ee5e239ccaf258dd356a70eb62a9ffad5333f70ee5fd2f5dc827 Jul 01 20:51:35.019709 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:35.019561 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-67485586b-pqcfn" event={"ID":"596568a3-e6ef-4595-8fdd-552ab2f99cc6","Type":"ContainerStarted","Data":"ea4d1faa5e2dd41fc6a3c34163ad13a55e3d5e93dce41333b82428008391e33e"} Jul 01 20:51:35.021730 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:35.021701 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" event={"ID":"6d498d32-13ed-4938-ab98-77a4137dbe07","Type":"ContainerStarted","Data":"4d6776350718ee5e239ccaf258dd356a70eb62a9ffad5333f70ee5fd2f5dc827"} Jul 01 20:51:38.034973 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:38.034878 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-67485586b-pqcfn" event={"ID":"596568a3-e6ef-4595-8fdd-552ab2f99cc6","Type":"ContainerStarted","Data":"d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf"} Jul 01 20:51:38.035390 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:38.034989 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:51:38.036540 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:38.036519 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" event={"ID":"6d498d32-13ed-4938-ab98-77a4137dbe07","Type":"ContainerStarted","Data":"3f8ed8c0ee531bab8a324852965057564c34dfc1465d5a710bb841799539ccc2"} Jul 01 20:51:38.036631 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:38.036565 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:51:38.101983 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:38.101934 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" podStartSLOduration=2.120164443 podStartE2EDuration="5.101920007s" totalImagesPullingTime="2.981755564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:51:38.100951224 +0000 UTC m=+282.709394099" watchObservedRunningTime="2026-07-01 20:51:38.101920007 +0000 UTC m=+282.710362881" Jul 01 20:51:38.102884 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:38.102855 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-67485586b-pqcfn" podStartSLOduration=1.527454236 podStartE2EDuration="5.102848256s" totalImagesPullingTime="3.57539402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:51:38.060247458 +0000 UTC m=+282.668690334" watchObservedRunningTime="2026-07-01 20:51:38.102848256 +0000 UTC m=+282.711291138" Jul 01 20:51:55.780491 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:55.780459 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 20:51:55.783077 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:55.783059 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 20:51:55.785005 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:51:55.784922 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 20:52:09.042555 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:09.042527 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-7d672" Jul 01 20:52:09.045608 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:09.045591 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:52:10.353227 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.353190 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-67485586b-pqcfn"] Jul 01 20:52:10.353646 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.353435 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-67485586b-pqcfn" podUID="596568a3-e6ef-4595-8fdd-552ab2f99cc6" containerName="manager" containerID="cri-o://d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf" gracePeriod=10 Jul 01 20:52:10.376322 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.376298 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-67485586b-4jpsp"] Jul 01 20:52:10.424920 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.424889 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:10.425047 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.424893 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-67485586b-4jpsp"] Jul 01 20:52:10.427812 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.427786 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e81df96-9892-41da-9516-0d51508b1f7a-cert\") pod \"kserve-controller-manager-67485586b-4jpsp\" (UID: \"5e81df96-9892-41da-9516-0d51508b1f7a\") " pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:10.427927 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.427863 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg8vj\" (UniqueName: \"kubernetes.io/projected/5e81df96-9892-41da-9516-0d51508b1f7a-kube-api-access-rg8vj\") pod \"kserve-controller-manager-67485586b-4jpsp\" (UID: \"5e81df96-9892-41da-9516-0d51508b1f7a\") " pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:10.528720 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.528687 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e81df96-9892-41da-9516-0d51508b1f7a-cert\") pod \"kserve-controller-manager-67485586b-4jpsp\" (UID: \"5e81df96-9892-41da-9516-0d51508b1f7a\") " pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:10.528878 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.528740 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg8vj\" (UniqueName: \"kubernetes.io/projected/5e81df96-9892-41da-9516-0d51508b1f7a-kube-api-access-rg8vj\") pod \"kserve-controller-manager-67485586b-4jpsp\" (UID: \"5e81df96-9892-41da-9516-0d51508b1f7a\") " pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:10.531195 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.531171 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e81df96-9892-41da-9516-0d51508b1f7a-cert\") pod \"kserve-controller-manager-67485586b-4jpsp\" (UID: \"5e81df96-9892-41da-9516-0d51508b1f7a\") " pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:10.536765 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.536739 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg8vj\" (UniqueName: \"kubernetes.io/projected/5e81df96-9892-41da-9516-0d51508b1f7a-kube-api-access-rg8vj\") pod \"kserve-controller-manager-67485586b-4jpsp\" (UID: \"5e81df96-9892-41da-9516-0d51508b1f7a\") " pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:10.624488 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.624431 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:52:10.629643 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.629627 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ks7b6\" (UniqueName: \"kubernetes.io/projected/596568a3-e6ef-4595-8fdd-552ab2f99cc6-kube-api-access-ks7b6\") pod \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\" (UID: \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\") " Jul 01 20:52:10.629737 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.629725 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/596568a3-e6ef-4595-8fdd-552ab2f99cc6-cert\") pod \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\" (UID: \"596568a3-e6ef-4595-8fdd-552ab2f99cc6\") " Jul 01 20:52:10.631749 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.631712 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596568a3-e6ef-4595-8fdd-552ab2f99cc6-kube-api-access-ks7b6" (OuterVolumeSpecName: "kube-api-access-ks7b6") pod "596568a3-e6ef-4595-8fdd-552ab2f99cc6" (UID: "596568a3-e6ef-4595-8fdd-552ab2f99cc6"). InnerVolumeSpecName "kube-api-access-ks7b6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 20:52:10.631837 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.631759 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/596568a3-e6ef-4595-8fdd-552ab2f99cc6-cert" (OuterVolumeSpecName: "cert") pod "596568a3-e6ef-4595-8fdd-552ab2f99cc6" (UID: "596568a3-e6ef-4595-8fdd-552ab2f99cc6"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 20:52:10.730249 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.730217 2558 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/596568a3-e6ef-4595-8fdd-552ab2f99cc6-cert\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:52:10.730249 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.730248 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks7b6\" (UniqueName: \"kubernetes.io/projected/596568a3-e6ef-4595-8fdd-552ab2f99cc6-kube-api-access-ks7b6\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:52:10.807663 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.807632 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:10.951927 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.951902 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-67485586b-4jpsp"] Jul 01 20:52:10.953785 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:52:10.953759 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e81df96_9892_41da_9516_0d51508b1f7a.slice/crio-6355c393b3cb5c6c6f4b8121651c2876891cfd3f6b2e7daef295ad6e3c485cc6 WatchSource:0}: Error finding container 6355c393b3cb5c6c6f4b8121651c2876891cfd3f6b2e7daef295ad6e3c485cc6: Status 404 returned error can't find the container with id 6355c393b3cb5c6c6f4b8121651c2876891cfd3f6b2e7daef295ad6e3c485cc6 Jul 01 20:52:10.954986 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:10.954969 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 20:52:11.167696 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.167609 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-67485586b-4jpsp" event={"ID":"5e81df96-9892-41da-9516-0d51508b1f7a","Type":"ContainerStarted","Data":"6355c393b3cb5c6c6f4b8121651c2876891cfd3f6b2e7daef295ad6e3c485cc6"} Jul 01 20:52:11.168853 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.168826 2558 generic.go:356] "Generic (PLEG): container finished" podID="596568a3-e6ef-4595-8fdd-552ab2f99cc6" containerID="d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf" exitCode=0 Jul 01 20:52:11.168956 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.168875 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-67485586b-pqcfn" Jul 01 20:52:11.168956 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.168884 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-67485586b-pqcfn" event={"ID":"596568a3-e6ef-4595-8fdd-552ab2f99cc6","Type":"ContainerDied","Data":"d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf"} Jul 01 20:52:11.168956 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.168925 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-67485586b-pqcfn" event={"ID":"596568a3-e6ef-4595-8fdd-552ab2f99cc6","Type":"ContainerDied","Data":"ea4d1faa5e2dd41fc6a3c34163ad13a55e3d5e93dce41333b82428008391e33e"} Jul 01 20:52:11.168956 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.168948 2558 scope.go:117] "RemoveContainer" containerID="d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf" Jul 01 20:52:11.177191 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.177177 2558 scope.go:117] "RemoveContainer" containerID="d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf" Jul 01 20:52:11.177453 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:52:11.177435 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf\": container with ID starting with d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf not found: ID does not exist" containerID="d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf" Jul 01 20:52:11.177513 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.177460 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf"} err="failed to get container status \"d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf\": rpc error: code = NotFound desc = could not find container \"d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf\": container with ID starting with d8a66a2c0db0baaed3e5e1e228c7df3c96959fa00568ae0cda91b1a8eef77baf not found: ID does not exist" Jul 01 20:52:11.188611 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.188591 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-67485586b-pqcfn"] Jul 01 20:52:11.192502 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.192485 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-67485586b-pqcfn"] Jul 01 20:52:11.843389 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:11.843355 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="596568a3-e6ef-4595-8fdd-552ab2f99cc6" path="/var/lib/kubelet/pods/596568a3-e6ef-4595-8fdd-552ab2f99cc6/volumes" Jul 01 20:52:12.174403 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:12.174300 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-67485586b-4jpsp" event={"ID":"5e81df96-9892-41da-9516-0d51508b1f7a","Type":"ContainerStarted","Data":"f9a0438b53cfd6912584e7534277556c02b612d57f0270cd940c3754ee0a6a86"} Jul 01 20:52:12.174403 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:12.174363 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:12.190800 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:12.190755 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-67485586b-4jpsp" podStartSLOduration=1.7004875720000001 podStartE2EDuration="2.190739237s" totalImagesPullingTime="490.251665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:52:12.189769817 +0000 UTC m=+316.798212705" watchObservedRunningTime="2026-07-01 20:52:12.190739237 +0000 UTC m=+316.799182111" Jul 01 20:52:43.185591 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:43.185561 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-67485586b-4jpsp" Jul 01 20:52:44.116659 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.116622 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-pn8dm"] Jul 01 20:52:44.116924 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.116912 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="596568a3-e6ef-4595-8fdd-552ab2f99cc6" containerName="manager" Jul 01 20:52:44.116976 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.116926 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="596568a3-e6ef-4595-8fdd-552ab2f99cc6" containerName="manager" Jul 01 20:52:44.117011 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.116980 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="596568a3-e6ef-4595-8fdd-552ab2f99cc6" containerName="manager" Jul 01 20:52:44.120049 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.120032 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:44.122302 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.122276 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jul 01 20:52:44.122419 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.122352 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-7qbgt\"" Jul 01 20:52:44.130022 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.130000 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-pn8dm"] Jul 01 20:52:44.133215 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.133194 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-j6ztm"] Jul 01 20:52:44.136270 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.136241 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:44.138287 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.138246 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jul 01 20:52:44.138347 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.138249 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-jjrkc\"" Jul 01 20:52:44.146561 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.146540 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-j6ztm"] Jul 01 20:52:44.194481 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.194446 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9gw5\" (UniqueName: \"kubernetes.io/projected/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-kube-api-access-p9gw5\") pod \"model-serving-api-fd65d7d6-pn8dm\" (UID: \"2ac30cbf-35b3-4280-9df0-7fd99197ce8e\") " pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:44.194481 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.194484 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmf7r\" (UniqueName: \"kubernetes.io/projected/2c34797a-6aef-4abf-8d85-141aab48f601-kube-api-access-zmf7r\") pod \"odh-model-controller-84b54b7f97-j6ztm\" (UID: \"2c34797a-6aef-4abf-8d85-141aab48f601\") " pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:44.194880 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.194546 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-tls-certs\") pod \"model-serving-api-fd65d7d6-pn8dm\" (UID: \"2ac30cbf-35b3-4280-9df0-7fd99197ce8e\") " pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:44.194880 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.194601 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c34797a-6aef-4abf-8d85-141aab48f601-cert\") pod \"odh-model-controller-84b54b7f97-j6ztm\" (UID: \"2c34797a-6aef-4abf-8d85-141aab48f601\") " pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:44.295188 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.295145 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c34797a-6aef-4abf-8d85-141aab48f601-cert\") pod \"odh-model-controller-84b54b7f97-j6ztm\" (UID: \"2c34797a-6aef-4abf-8d85-141aab48f601\") " pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:44.295391 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.295215 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9gw5\" (UniqueName: \"kubernetes.io/projected/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-kube-api-access-p9gw5\") pod \"model-serving-api-fd65d7d6-pn8dm\" (UID: \"2ac30cbf-35b3-4280-9df0-7fd99197ce8e\") " pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:44.295391 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.295253 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmf7r\" (UniqueName: \"kubernetes.io/projected/2c34797a-6aef-4abf-8d85-141aab48f601-kube-api-access-zmf7r\") pod \"odh-model-controller-84b54b7f97-j6ztm\" (UID: \"2c34797a-6aef-4abf-8d85-141aab48f601\") " pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:44.295391 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.295319 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-tls-certs\") pod \"model-serving-api-fd65d7d6-pn8dm\" (UID: \"2ac30cbf-35b3-4280-9df0-7fd99197ce8e\") " pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:44.295391 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:52:44.295341 2558 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 01 20:52:44.295599 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:52:44.295418 2558 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 01 20:52:44.295599 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:52:44.295451 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c34797a-6aef-4abf-8d85-141aab48f601-cert podName:2c34797a-6aef-4abf-8d85-141aab48f601 nodeName:}" failed. No retries permitted until 2026-07-01 20:52:44.795428035 +0000 UTC m=+349.403870889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c34797a-6aef-4abf-8d85-141aab48f601-cert") pod "odh-model-controller-84b54b7f97-j6ztm" (UID: "2c34797a-6aef-4abf-8d85-141aab48f601") : secret "odh-model-controller-webhook-cert" not found Jul 01 20:52:44.295599 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:52:44.295470 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-tls-certs podName:2ac30cbf-35b3-4280-9df0-7fd99197ce8e nodeName:}" failed. No retries permitted until 2026-07-01 20:52:44.7954628 +0000 UTC m=+349.403905655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-tls-certs") pod "model-serving-api-fd65d7d6-pn8dm" (UID: "2ac30cbf-35b3-4280-9df0-7fd99197ce8e") : secret "model-serving-api-tls" not found Jul 01 20:52:44.306277 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.306230 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmf7r\" (UniqueName: \"kubernetes.io/projected/2c34797a-6aef-4abf-8d85-141aab48f601-kube-api-access-zmf7r\") pod \"odh-model-controller-84b54b7f97-j6ztm\" (UID: \"2c34797a-6aef-4abf-8d85-141aab48f601\") " pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:44.306929 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.306911 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9gw5\" (UniqueName: \"kubernetes.io/projected/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-kube-api-access-p9gw5\") pod \"model-serving-api-fd65d7d6-pn8dm\" (UID: \"2ac30cbf-35b3-4280-9df0-7fd99197ce8e\") " pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:44.799693 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.799659 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c34797a-6aef-4abf-8d85-141aab48f601-cert\") pod \"odh-model-controller-84b54b7f97-j6ztm\" (UID: \"2c34797a-6aef-4abf-8d85-141aab48f601\") " pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:44.799871 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.799730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-tls-certs\") pod \"model-serving-api-fd65d7d6-pn8dm\" (UID: \"2ac30cbf-35b3-4280-9df0-7fd99197ce8e\") " pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:44.802240 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.802211 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c34797a-6aef-4abf-8d85-141aab48f601-cert\") pod \"odh-model-controller-84b54b7f97-j6ztm\" (UID: \"2c34797a-6aef-4abf-8d85-141aab48f601\") " pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:44.802240 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:44.802231 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ac30cbf-35b3-4280-9df0-7fd99197ce8e-tls-certs\") pod \"model-serving-api-fd65d7d6-pn8dm\" (UID: \"2ac30cbf-35b3-4280-9df0-7fd99197ce8e\") " pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:45.029930 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:45.029897 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:45.046176 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:45.046150 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:45.168488 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:45.168333 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-pn8dm"] Jul 01 20:52:45.171414 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:52:45.171372 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac30cbf_35b3_4280_9df0_7fd99197ce8e.slice/crio-57c5954456a236683ab996433e06baca9f99bad176c2c8f8e3971647ada279be WatchSource:0}: Error finding container 57c5954456a236683ab996433e06baca9f99bad176c2c8f8e3971647ada279be: Status 404 returned error can't find the container with id 57c5954456a236683ab996433e06baca9f99bad176c2c8f8e3971647ada279be Jul 01 20:52:45.191186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:45.191163 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-j6ztm"] Jul 01 20:52:45.193081 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:52:45.193054 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c34797a_6aef_4abf_8d85_141aab48f601.slice/crio-3b75a19df152025d880e0583afabd78e080772eb4ebd4a94d8423701c3c2c192 WatchSource:0}: Error finding container 3b75a19df152025d880e0583afabd78e080772eb4ebd4a94d8423701c3c2c192: Status 404 returned error can't find the container with id 3b75a19df152025d880e0583afabd78e080772eb4ebd4a94d8423701c3c2c192 Jul 01 20:52:45.311912 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:45.311811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-j6ztm" event={"ID":"2c34797a-6aef-4abf-8d85-141aab48f601","Type":"ContainerStarted","Data":"3b75a19df152025d880e0583afabd78e080772eb4ebd4a94d8423701c3c2c192"} Jul 01 20:52:45.313029 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:45.313007 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-pn8dm" event={"ID":"2ac30cbf-35b3-4280-9df0-7fd99197ce8e","Type":"ContainerStarted","Data":"57c5954456a236683ab996433e06baca9f99bad176c2c8f8e3971647ada279be"} Jul 01 20:52:49.333934 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:49.333876 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-j6ztm" event={"ID":"2c34797a-6aef-4abf-8d85-141aab48f601","Type":"ContainerStarted","Data":"d6e67355cbf6b0ea177c0deae32ef417a3e932718ceda7d3e36538763fc45252"} Jul 01 20:52:49.334421 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:49.333962 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:52:49.335690 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:49.335664 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-pn8dm" event={"ID":"2ac30cbf-35b3-4280-9df0-7fd99197ce8e","Type":"ContainerStarted","Data":"85048a0dc4e75f9c30f84fc7af2a1860454556f220aca0e10c5ac86fec087b93"} Jul 01 20:52:49.335795 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:49.335723 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:52:49.353212 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:49.353152 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-j6ztm" podStartSLOduration=1.521797164 podStartE2EDuration="5.353135105s" totalImagesPullingTime="3.831337941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:52:49.351147339 +0000 UTC m=+353.959590214" watchObservedRunningTime="2026-07-01 20:52:49.353135105 +0000 UTC m=+353.961577979" Jul 01 20:52:49.368938 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:52:49.368894 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-pn8dm" podStartSLOduration=1.522086703 podStartE2EDuration="5.368879111s" totalImagesPullingTime="3.846792408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:52:49.367547263 +0000 UTC m=+353.975990140" watchObservedRunningTime="2026-07-01 20:52:49.368879111 +0000 UTC m=+353.977321986" Jul 01 20:53:00.342397 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:00.342371 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-j6ztm" Jul 01 20:53:00.343928 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:00.343909 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-pn8dm" Jul 01 20:53:03.672874 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.672838 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-549ffd5ff4-n6rsd"] Jul 01 20:53:03.677676 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.677654 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.691592 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.691570 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549ffd5ff4-n6rsd"] Jul 01 20:53:03.757497 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.757465 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-trusted-ca-bundle\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.757640 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.757502 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-oauth-config\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.757640 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.757523 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-serving-cert\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.757640 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.757598 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-service-ca\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.757640 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.757635 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-config\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.757771 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.757665 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jd96\" (UniqueName: \"kubernetes.io/projected/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-kube-api-access-4jd96\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.757771 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.757691 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-oauth-serving-cert\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858068 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858032 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-config\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858198 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858084 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jd96\" (UniqueName: \"kubernetes.io/projected/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-kube-api-access-4jd96\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858198 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-oauth-serving-cert\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858198 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858144 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-trusted-ca-bundle\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858198 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858167 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-oauth-config\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858198 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858194 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-serving-cert\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858478 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858231 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-service-ca\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858894 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858845 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-config\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858894 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858858 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-service-ca\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.858894 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.858845 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-oauth-serving-cert\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.859247 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.859226 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-trusted-ca-bundle\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.860656 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.860637 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-serving-cert\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.860733 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.860667 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-console-oauth-config\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.866771 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.866748 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jd96\" (UniqueName: \"kubernetes.io/projected/5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1-kube-api-access-4jd96\") pod \"console-549ffd5ff4-n6rsd\" (UID: \"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1\") " pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:03.986888 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:03.986812 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:04.118093 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:04.118066 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549ffd5ff4-n6rsd"] Jul 01 20:53:04.120530 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:53:04.120503 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c77a363_eaee_4df8_ada1_2a6b6d4f6dc1.slice/crio-a1d042bebcd11eb1d237e7f58c87d39bfedf2bb09a7f9a6d7c9764c7a4df8692 WatchSource:0}: Error finding container a1d042bebcd11eb1d237e7f58c87d39bfedf2bb09a7f9a6d7c9764c7a4df8692: Status 404 returned error can't find the container with id a1d042bebcd11eb1d237e7f58c87d39bfedf2bb09a7f9a6d7c9764c7a4df8692 Jul 01 20:53:04.397298 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:04.397247 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549ffd5ff4-n6rsd" event={"ID":"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1","Type":"ContainerStarted","Data":"b7a960e34a158b0efc25618f04e47a679816868e2bf0340d991ba5af311b5488"} Jul 01 20:53:04.397487 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:04.397343 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549ffd5ff4-n6rsd" event={"ID":"5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1","Type":"ContainerStarted","Data":"a1d042bebcd11eb1d237e7f58c87d39bfedf2bb09a7f9a6d7c9764c7a4df8692"} Jul 01 20:53:04.414863 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:04.414813 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-549ffd5ff4-n6rsd" podStartSLOduration=1.414798711 podStartE2EDuration="1.414798711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:53:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:53:04.413955909 +0000 UTC m=+369.022398786" watchObservedRunningTime="2026-07-01 20:53:04.414798711 +0000 UTC m=+369.023241586" Jul 01 20:53:13.987955 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:13.987912 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:13.988476 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:13.987993 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:13.992782 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:13.992761 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:14.441689 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:14.441662 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-549ffd5ff4-n6rsd" Jul 01 20:53:14.486421 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:14.486392 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cd7d5f7f-dq4ws"] Jul 01 20:53:39.507180 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.507139 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76cd7d5f7f-dq4ws" podUID="e19ea2ca-4447-4515-ba34-480fa85ea699" containerName="console" containerID="cri-o://1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6" gracePeriod=15 Jul 01 20:53:39.751787 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.751761 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:53:39.847429 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847405 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-oauth-serving-cert\") pod \"e19ea2ca-4447-4515-ba34-480fa85ea699\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " Jul 01 20:53:39.847597 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847457 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-oauth-config\") pod \"e19ea2ca-4447-4515-ba34-480fa85ea699\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " Jul 01 20:53:39.847597 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847528 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw2zw\" (UniqueName: \"kubernetes.io/projected/e19ea2ca-4447-4515-ba34-480fa85ea699-kube-api-access-nw2zw\") pod \"e19ea2ca-4447-4515-ba34-480fa85ea699\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " Jul 01 20:53:39.847597 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847569 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-trusted-ca-bundle\") pod \"e19ea2ca-4447-4515-ba34-480fa85ea699\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " Jul 01 20:53:39.847753 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847602 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-service-ca\") pod \"e19ea2ca-4447-4515-ba34-480fa85ea699\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " Jul 01 20:53:39.847753 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847630 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-serving-cert\") pod \"e19ea2ca-4447-4515-ba34-480fa85ea699\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " Jul 01 20:53:39.847753 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847675 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-console-config\") pod \"e19ea2ca-4447-4515-ba34-480fa85ea699\" (UID: \"e19ea2ca-4447-4515-ba34-480fa85ea699\") " Jul 01 20:53:39.847935 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847812 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e19ea2ca-4447-4515-ba34-480fa85ea699" (UID: "e19ea2ca-4447-4515-ba34-480fa85ea699"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:53:39.848055 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.847994 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-service-ca" (OuterVolumeSpecName: "service-ca") pod "e19ea2ca-4447-4515-ba34-480fa85ea699" (UID: "e19ea2ca-4447-4515-ba34-480fa85ea699"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:53:39.848055 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.848028 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e19ea2ca-4447-4515-ba34-480fa85ea699" (UID: "e19ea2ca-4447-4515-ba34-480fa85ea699"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:53:39.848447 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.848356 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-console-config" (OuterVolumeSpecName: "console-config") pod "e19ea2ca-4447-4515-ba34-480fa85ea699" (UID: "e19ea2ca-4447-4515-ba34-480fa85ea699"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:53:39.849824 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.849800 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e19ea2ca-4447-4515-ba34-480fa85ea699" (UID: "e19ea2ca-4447-4515-ba34-480fa85ea699"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 20:53:39.849824 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.849814 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e19ea2ca-4447-4515-ba34-480fa85ea699-kube-api-access-nw2zw" (OuterVolumeSpecName: "kube-api-access-nw2zw") pod "e19ea2ca-4447-4515-ba34-480fa85ea699" (UID: "e19ea2ca-4447-4515-ba34-480fa85ea699"). InnerVolumeSpecName "kube-api-access-nw2zw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 20:53:39.849937 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.849818 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e19ea2ca-4447-4515-ba34-480fa85ea699" (UID: "e19ea2ca-4447-4515-ba34-480fa85ea699"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 20:53:39.948399 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.948363 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-oauth-serving-cert\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:53:39.948399 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.948393 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-oauth-config\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:53:39.948399 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.948402 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw2zw\" (UniqueName: \"kubernetes.io/projected/e19ea2ca-4447-4515-ba34-480fa85ea699-kube-api-access-nw2zw\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:53:39.948605 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.948414 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-trusted-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:53:39.948605 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.948430 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-service-ca\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:53:39.948605 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.948440 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e19ea2ca-4447-4515-ba34-480fa85ea699-console-serving-cert\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:53:39.948605 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:39.948447 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e19ea2ca-4447-4515-ba34-480fa85ea699-console-config\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:53:40.541957 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.541922 2558 generic.go:356] "Generic (PLEG): container finished" podID="e19ea2ca-4447-4515-ba34-480fa85ea699" containerID="1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6" exitCode=0 Jul 01 20:53:40.542407 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.541970 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cd7d5f7f-dq4ws" Jul 01 20:53:40.542407 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.541975 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cd7d5f7f-dq4ws" event={"ID":"e19ea2ca-4447-4515-ba34-480fa85ea699","Type":"ContainerDied","Data":"1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6"} Jul 01 20:53:40.542407 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.542024 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cd7d5f7f-dq4ws" event={"ID":"e19ea2ca-4447-4515-ba34-480fa85ea699","Type":"ContainerDied","Data":"92a037b030ddcaea6a38f6a6aedff7e3cfd1d0916b0e480aac5b9ea73a08250f"} Jul 01 20:53:40.542407 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.542052 2558 scope.go:117] "RemoveContainer" containerID="1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6" Jul 01 20:53:40.550292 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.550270 2558 scope.go:117] "RemoveContainer" containerID="1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6" Jul 01 20:53:40.550545 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:53:40.550526 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6\": container with ID starting with 1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6 not found: ID does not exist" containerID="1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6" Jul 01 20:53:40.550606 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.550554 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6"} err="failed to get container status \"1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6\": rpc error: code = NotFound desc = could not find container \"1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6\": container with ID starting with 1966a33a3b35c6b2912bc70d0785be9f9a1e5ca79fb8d4fb9dc4edbbfa982af6 not found: ID does not exist" Jul 01 20:53:40.564667 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.564645 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cd7d5f7f-dq4ws"] Jul 01 20:53:40.566817 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:40.566796 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76cd7d5f7f-dq4ws"] Jul 01 20:53:41.843463 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:53:41.843425 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e19ea2ca-4447-4515-ba34-480fa85ea699" path="/var/lib/kubelet/pods/e19ea2ca-4447-4515-ba34-480fa85ea699/volumes" Jul 01 20:56:22.053657 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.053625 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd"] Jul 01 20:56:22.054111 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.053937 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e19ea2ca-4447-4515-ba34-480fa85ea699" containerName="console" Jul 01 20:56:22.054111 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.053949 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e19ea2ca-4447-4515-ba34-480fa85ea699" containerName="console" Jul 01 20:56:22.054111 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.054015 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e19ea2ca-4447-4515-ba34-480fa85ea699" containerName="console" Jul 01 20:56:22.056933 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.056915 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:22.059074 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.059047 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-25c51-kube-rbac-proxy-sar-config\"" Jul 01 20:56:22.059164 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.059051 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 20:56:22.059203 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.059189 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-25c51-serving-cert\"" Jul 01 20:56:22.059768 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.059750 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vwmkn\"" Jul 01 20:56:22.066348 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.066329 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd"] Jul 01 20:56:22.206588 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.206560 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad8dff79-1e91-4a85-8400-9d86376cb3d5-proxy-tls\") pod \"switch-graph-25c51-56c67f6cb4-wlzqd\" (UID: \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\") " pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:22.206752 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.206608 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad8dff79-1e91-4a85-8400-9d86376cb3d5-openshift-service-ca-bundle\") pod \"switch-graph-25c51-56c67f6cb4-wlzqd\" (UID: \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\") " pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:22.307166 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.307089 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad8dff79-1e91-4a85-8400-9d86376cb3d5-openshift-service-ca-bundle\") pod \"switch-graph-25c51-56c67f6cb4-wlzqd\" (UID: \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\") " pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:22.307166 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.307149 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad8dff79-1e91-4a85-8400-9d86376cb3d5-proxy-tls\") pod \"switch-graph-25c51-56c67f6cb4-wlzqd\" (UID: \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\") " pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:22.307727 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.307695 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad8dff79-1e91-4a85-8400-9d86376cb3d5-openshift-service-ca-bundle\") pod \"switch-graph-25c51-56c67f6cb4-wlzqd\" (UID: \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\") " pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:22.309526 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.309508 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad8dff79-1e91-4a85-8400-9d86376cb3d5-proxy-tls\") pod \"switch-graph-25c51-56c67f6cb4-wlzqd\" (UID: \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\") " pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:22.367007 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.366978 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:22.495721 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:22.495690 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd"] Jul 01 20:56:22.498114 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:56:22.498085 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8dff79_1e91_4a85_8400_9d86376cb3d5.slice/crio-08d76e67f0fa3347f1309f880ca0432fa003c2ceed4debb5bfa045847f093178 WatchSource:0}: Error finding container 08d76e67f0fa3347f1309f880ca0432fa003c2ceed4debb5bfa045847f093178: Status 404 returned error can't find the container with id 08d76e67f0fa3347f1309f880ca0432fa003c2ceed4debb5bfa045847f093178 Jul 01 20:56:23.189995 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:23.189962 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" event={"ID":"ad8dff79-1e91-4a85-8400-9d86376cb3d5","Type":"ContainerStarted","Data":"08d76e67f0fa3347f1309f880ca0432fa003c2ceed4debb5bfa045847f093178"} Jul 01 20:56:25.200471 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:25.200437 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" event={"ID":"ad8dff79-1e91-4a85-8400-9d86376cb3d5","Type":"ContainerStarted","Data":"858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51"} Jul 01 20:56:25.200939 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:25.200560 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:25.216280 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:25.216218 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" podStartSLOduration=1.011165081 podStartE2EDuration="3.21620289s" totalImagesPullingTime="2.205037809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:56:25.214973462 +0000 UTC m=+569.823416337" watchObservedRunningTime="2026-07-01 20:56:25.21620289 +0000 UTC m=+569.824645761" Jul 01 20:56:31.209695 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:31.209662 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:32.173074 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:32.173029 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd"] Jul 01 20:56:32.173278 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:32.173233 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" containerID="cri-o://858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51" gracePeriod=30 Jul 01 20:56:36.209013 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:36.208970 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:56:41.209015 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:41.208972 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:56:46.208762 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:46.208726 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:56:46.209225 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:46.208825 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:56:51.209023 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:51.208976 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:56:52.066127 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.066094 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx"] Jul 01 20:56:52.069388 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.069368 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:52.071774 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.071753 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Jul 01 20:56:52.071895 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.071877 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Jul 01 20:56:52.077103 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.077076 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx"] Jul 01 20:56:52.152334 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.152307 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls\") pod \"model-chainer-794cbf94f-6krlx\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:52.152462 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.152337 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64ef75e8-c42f-40bb-9494-61c574ce3025-openshift-service-ca-bundle\") pod \"model-chainer-794cbf94f-6krlx\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:52.253369 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.253333 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls\") pod \"model-chainer-794cbf94f-6krlx\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:52.253369 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.253375 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64ef75e8-c42f-40bb-9494-61c574ce3025-openshift-service-ca-bundle\") pod \"model-chainer-794cbf94f-6krlx\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:52.253775 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:56:52.253481 2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found Jul 01 20:56:52.253775 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:56:52.253545 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls podName:64ef75e8-c42f-40bb-9494-61c574ce3025 nodeName:}" failed. No retries permitted until 2026-07-01 20:56:52.753529101 +0000 UTC m=+597.361971953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls") pod "model-chainer-794cbf94f-6krlx" (UID: "64ef75e8-c42f-40bb-9494-61c574ce3025") : secret "model-chainer-serving-cert" not found Jul 01 20:56:52.253985 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.253969 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64ef75e8-c42f-40bb-9494-61c574ce3025-openshift-service-ca-bundle\") pod \"model-chainer-794cbf94f-6krlx\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:52.756472 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.756435 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls\") pod \"model-chainer-794cbf94f-6krlx\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:52.758893 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.758866 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls\") pod \"model-chainer-794cbf94f-6krlx\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:52.980022 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:52.979990 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:53.113290 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:53.109332 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx"] Jul 01 20:56:53.321625 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:53.321590 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" event={"ID":"64ef75e8-c42f-40bb-9494-61c574ce3025","Type":"ContainerStarted","Data":"ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327"} Jul 01 20:56:53.322083 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:53.321637 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" event={"ID":"64ef75e8-c42f-40bb-9494-61c574ce3025","Type":"ContainerStarted","Data":"84461a8ba7d22f2fd096574ab2425b2ef73c394ff6b7a0f4779f3139e1fd7bc7"} Jul 01 20:56:53.322083 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:53.321752 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:56:53.337698 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:53.337656 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" podStartSLOduration=1.337643436 podStartE2EDuration="1.337643436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:56:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:56:53.33616769 +0000 UTC m=+597.944610566" watchObservedRunningTime="2026-07-01 20:56:53.337643436 +0000 UTC m=+597.946086310" Jul 01 20:56:55.809689 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:55.809665 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 20:56:55.810081 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:55.810006 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 20:56:56.209052 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:56.209016 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:56:59.330753 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:56:59.330719 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:57:01.208991 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:01.208952 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:57:02.188016 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.187979 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx"] Jul 01 20:57:02.188315 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.188195 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" containerID="cri-o://ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327" gracePeriod=30 Jul 01 20:57:02.340620 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.340568 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:57:02.358245 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.358214 2558 generic.go:356] "Generic (PLEG): container finished" podID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerID="858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51" exitCode=0 Jul 01 20:57:02.358407 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.358280 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" Jul 01 20:57:02.358407 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.358301 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" event={"ID":"ad8dff79-1e91-4a85-8400-9d86376cb3d5","Type":"ContainerDied","Data":"858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51"} Jul 01 20:57:02.358407 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.358350 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd" event={"ID":"ad8dff79-1e91-4a85-8400-9d86376cb3d5","Type":"ContainerDied","Data":"08d76e67f0fa3347f1309f880ca0432fa003c2ceed4debb5bfa045847f093178"} Jul 01 20:57:02.358407 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.358371 2558 scope.go:117] "RemoveContainer" containerID="858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51" Jul 01 20:57:02.366216 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.366200 2558 scope.go:117] "RemoveContainer" containerID="858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51" Jul 01 20:57:02.366515 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:57:02.366496 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51\": container with ID starting with 858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51 not found: ID does not exist" containerID="858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51" Jul 01 20:57:02.366577 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.366530 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51"} err="failed to get container status \"858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51\": rpc error: code = NotFound desc = could not find container \"858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51\": container with ID starting with 858ce95fd397c08a3e2ce38091d1ae6b477173c2f1a4192bd83f23c8577b0b51 not found: ID does not exist" Jul 01 20:57:02.437534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.437465 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad8dff79-1e91-4a85-8400-9d86376cb3d5-proxy-tls\") pod \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\" (UID: \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\") " Jul 01 20:57:02.437534 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.437500 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad8dff79-1e91-4a85-8400-9d86376cb3d5-openshift-service-ca-bundle\") pod \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\" (UID: \"ad8dff79-1e91-4a85-8400-9d86376cb3d5\") " Jul 01 20:57:02.437901 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.437876 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad8dff79-1e91-4a85-8400-9d86376cb3d5-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ad8dff79-1e91-4a85-8400-9d86376cb3d5" (UID: "ad8dff79-1e91-4a85-8400-9d86376cb3d5"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:57:02.439708 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.439685 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad8dff79-1e91-4a85-8400-9d86376cb3d5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ad8dff79-1e91-4a85-8400-9d86376cb3d5" (UID: "ad8dff79-1e91-4a85-8400-9d86376cb3d5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 20:57:02.538657 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.538627 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ad8dff79-1e91-4a85-8400-9d86376cb3d5-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:57:02.538657 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.538656 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad8dff79-1e91-4a85-8400-9d86376cb3d5-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:57:02.690440 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.690380 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd"] Jul 01 20:57:02.707523 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:02.707495 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-25c51-56c67f6cb4-wlzqd"] Jul 01 20:57:03.843945 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:03.843901 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" path="/var/lib/kubelet/pods/ad8dff79-1e91-4a85-8400-9d86376cb3d5/volumes" Jul 01 20:57:04.329912 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:04.329875 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:57:09.329409 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:09.329372 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:57:14.329512 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:14.329464 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:57:14.329903 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:14.329562 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:57:19.329330 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:19.329224 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:57:24.328760 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:24.328716 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:57:29.329538 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:29.329502 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 20:57:32.382186 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.382156 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:57:32.473708 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.473640 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64ef75e8-c42f-40bb-9494-61c574ce3025-openshift-service-ca-bundle\") pod \"64ef75e8-c42f-40bb-9494-61c574ce3025\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " Jul 01 20:57:32.473708 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.473676 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls\") pod \"64ef75e8-c42f-40bb-9494-61c574ce3025\" (UID: \"64ef75e8-c42f-40bb-9494-61c574ce3025\") " Jul 01 20:57:32.474004 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.473981 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ef75e8-c42f-40bb-9494-61c574ce3025-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "64ef75e8-c42f-40bb-9494-61c574ce3025" (UID: "64ef75e8-c42f-40bb-9494-61c574ce3025"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 20:57:32.475795 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.475771 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "64ef75e8-c42f-40bb-9494-61c574ce3025" (UID: "64ef75e8-c42f-40bb-9494-61c574ce3025"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 20:57:32.480139 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.480115 2558 generic.go:356] "Generic (PLEG): container finished" podID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerID="ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327" exitCode=137 Jul 01 20:57:32.480239 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.480158 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" Jul 01 20:57:32.480239 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.480162 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" event={"ID":"64ef75e8-c42f-40bb-9494-61c574ce3025","Type":"ContainerDied","Data":"ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327"} Jul 01 20:57:32.480239 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.480203 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx" event={"ID":"64ef75e8-c42f-40bb-9494-61c574ce3025","Type":"ContainerDied","Data":"84461a8ba7d22f2fd096574ab2425b2ef73c394ff6b7a0f4779f3139e1fd7bc7"} Jul 01 20:57:32.480239 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.480224 2558 scope.go:117] "RemoveContainer" containerID="ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327" Jul 01 20:57:32.494558 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.494488 2558 scope.go:117] "RemoveContainer" containerID="ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327" Jul 01 20:57:32.494947 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:57:32.494926 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327\": container with ID starting with ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327 not found: ID does not exist" containerID="ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327" Jul 01 20:57:32.495034 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.494958 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327"} err="failed to get container status \"ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327\": rpc error: code = NotFound desc = could not find container \"ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327\": container with ID starting with ba76b3f309d4522d1c396bf70c33862edd168f574b0754cf4b21d7db5a835327 not found: ID does not exist" Jul 01 20:57:32.511387 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.511366 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx"] Jul 01 20:57:32.514935 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.514916 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-794cbf94f-6krlx"] Jul 01 20:57:32.574859 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.574826 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64ef75e8-c42f-40bb-9494-61c574ce3025-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:57:32.574859 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:32.574853 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ef75e8-c42f-40bb-9494-61c574ce3025-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 20:57:33.844223 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:33.844190 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" path="/var/lib/kubelet/pods/64ef75e8-c42f-40bb-9494-61c574ce3025/volumes" Jul 01 20:57:42.404175 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.404140 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4"] Jul 01 20:57:42.404552 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.404476 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" Jul 01 20:57:42.404552 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.404488 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" Jul 01 20:57:42.404552 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.404499 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" Jul 01 20:57:42.404552 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.404504 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" Jul 01 20:57:42.404674 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.404565 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="64ef75e8-c42f-40bb-9494-61c574ce3025" containerName="model-chainer" Jul 01 20:57:42.404674 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.404573 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad8dff79-1e91-4a85-8400-9d86376cb3d5" containerName="switch-graph-25c51" Jul 01 20:57:42.407191 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.407172 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:42.409374 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.409354 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-f347e-kube-rbac-proxy-sar-config\"" Jul 01 20:57:42.409472 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.409455 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 20:57:42.409598 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.409582 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-f347e-serving-cert\"" Jul 01 20:57:42.409666 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.409632 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vwmkn\"" Jul 01 20:57:42.418084 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.418063 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4"] Jul 01 20:57:42.447750 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.447721 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-openshift-service-ca-bundle\") pod \"switch-graph-f347e-6f4bb9d8fd-f5wl4\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:42.447879 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.447793 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls\") pod \"switch-graph-f347e-6f4bb9d8fd-f5wl4\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:42.548690 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.548656 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls\") pod \"switch-graph-f347e-6f4bb9d8fd-f5wl4\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:42.548858 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.548708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-openshift-service-ca-bundle\") pod \"switch-graph-f347e-6f4bb9d8fd-f5wl4\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:42.548858 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:57:42.548814 2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-f347e-serving-cert: secret "switch-graph-f347e-serving-cert" not found Jul 01 20:57:42.548952 ip-10-0-130-150 kubenswrapper[2558]: E0701 20:57:42.548901 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls podName:23cc18e6-9ac8-46b2-9b5e-c20702bbd904 nodeName:}" failed. No retries permitted until 2026-07-01 20:57:43.048881355 +0000 UTC m=+647.657324224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls") pod "switch-graph-f347e-6f4bb9d8fd-f5wl4" (UID: "23cc18e6-9ac8-46b2-9b5e-c20702bbd904") : secret "switch-graph-f347e-serving-cert" not found Jul 01 20:57:42.549439 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:42.549422 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-openshift-service-ca-bundle\") pod \"switch-graph-f347e-6f4bb9d8fd-f5wl4\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:43.052906 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:43.052871 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls\") pod \"switch-graph-f347e-6f4bb9d8fd-f5wl4\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:43.055338 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:43.055319 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls\") pod \"switch-graph-f347e-6f4bb9d8fd-f5wl4\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:43.317454 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:43.317428 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:43.443899 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:43.443875 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4"] Jul 01 20:57:43.446412 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:57:43.446382 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cc18e6_9ac8_46b2_9b5e_c20702bbd904.slice/crio-d3941b2e02c31bac5cd4a1477d65daa08b4b0495f76e151f676d9af29d90a49a WatchSource:0}: Error finding container d3941b2e02c31bac5cd4a1477d65daa08b4b0495f76e151f676d9af29d90a49a: Status 404 returned error can't find the container with id d3941b2e02c31bac5cd4a1477d65daa08b4b0495f76e151f676d9af29d90a49a Jul 01 20:57:43.527390 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:43.527350 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" event={"ID":"23cc18e6-9ac8-46b2-9b5e-c20702bbd904","Type":"ContainerStarted","Data":"ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59"} Jul 01 20:57:43.527526 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:43.527408 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" event={"ID":"23cc18e6-9ac8-46b2-9b5e-c20702bbd904","Type":"ContainerStarted","Data":"d3941b2e02c31bac5cd4a1477d65daa08b4b0495f76e151f676d9af29d90a49a"} Jul 01 20:57:43.527526 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:43.527473 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:57:43.544309 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:43.544243 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" podStartSLOduration=1.544228419 podStartE2EDuration="1.544228419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:57:43.542487903 +0000 UTC m=+648.150930779" watchObservedRunningTime="2026-07-01 20:57:43.544228419 +0000 UTC m=+648.152671294" Jul 01 20:57:49.537672 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:57:49.537644 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 20:58:02.373230 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.373195 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l"] Jul 01 20:58:02.376433 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.376410 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:02.381356 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.381336 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-a93e1-kube-rbac-proxy-sar-config\"" Jul 01 20:58:02.381456 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.381338 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-a93e1-serving-cert\"" Jul 01 20:58:02.387492 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.387470 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l"] Jul 01 20:58:02.505751 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.505718 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b90842f-4019-43ef-bc18-c15222f07ecc-proxy-tls\") pod \"sequence-graph-a93e1-84bdb6d745-qlk2l\" (UID: \"0b90842f-4019-43ef-bc18-c15222f07ecc\") " pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:02.505904 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.505772 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90842f-4019-43ef-bc18-c15222f07ecc-openshift-service-ca-bundle\") pod \"sequence-graph-a93e1-84bdb6d745-qlk2l\" (UID: \"0b90842f-4019-43ef-bc18-c15222f07ecc\") " pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:02.606139 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.606104 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b90842f-4019-43ef-bc18-c15222f07ecc-proxy-tls\") pod \"sequence-graph-a93e1-84bdb6d745-qlk2l\" (UID: \"0b90842f-4019-43ef-bc18-c15222f07ecc\") " pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:02.606328 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.606150 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90842f-4019-43ef-bc18-c15222f07ecc-openshift-service-ca-bundle\") pod \"sequence-graph-a93e1-84bdb6d745-qlk2l\" (UID: \"0b90842f-4019-43ef-bc18-c15222f07ecc\") " pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:02.606714 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.606696 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90842f-4019-43ef-bc18-c15222f07ecc-openshift-service-ca-bundle\") pod \"sequence-graph-a93e1-84bdb6d745-qlk2l\" (UID: \"0b90842f-4019-43ef-bc18-c15222f07ecc\") " pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:02.608588 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.608572 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b90842f-4019-43ef-bc18-c15222f07ecc-proxy-tls\") pod \"sequence-graph-a93e1-84bdb6d745-qlk2l\" (UID: \"0b90842f-4019-43ef-bc18-c15222f07ecc\") " pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:02.686250 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.686184 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:02.818491 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:02.818470 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l"] Jul 01 20:58:02.821103 ip-10-0-130-150 kubenswrapper[2558]: W0701 20:58:02.821076 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b90842f_4019_43ef_bc18_c15222f07ecc.slice/crio-38c17326791d03002ad08d3e25603db7e6084ea02f4d5a6b914ae29dc1b79e29 WatchSource:0}: Error finding container 38c17326791d03002ad08d3e25603db7e6084ea02f4d5a6b914ae29dc1b79e29: Status 404 returned error can't find the container with id 38c17326791d03002ad08d3e25603db7e6084ea02f4d5a6b914ae29dc1b79e29 Jul 01 20:58:03.611692 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:03.611658 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" event={"ID":"0b90842f-4019-43ef-bc18-c15222f07ecc","Type":"ContainerStarted","Data":"26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930"} Jul 01 20:58:03.612131 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:03.611707 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" event={"ID":"0b90842f-4019-43ef-bc18-c15222f07ecc","Type":"ContainerStarted","Data":"38c17326791d03002ad08d3e25603db7e6084ea02f4d5a6b914ae29dc1b79e29"} Jul 01 20:58:03.612131 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:03.611771 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 20:58:03.628064 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:03.628007 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" podStartSLOduration=1.627993709 podStartE2EDuration="1.627993709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:58:03.626788218 +0000 UTC m=+668.235231090" watchObservedRunningTime="2026-07-01 20:58:03.627993709 +0000 UTC m=+668.236436615" Jul 01 20:58:09.622903 ip-10-0-130-150 kubenswrapper[2558]: I0701 20:58:09.622868 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 21:01:55.837331 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:01:55.837235 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:01:55.839563 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:01:55.839542 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:05:57.146279 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:05:57.146224 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4"] Jul 01 21:05:57.146831 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:05:57.146501 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" containerID="cri-o://ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59" gracePeriod=30 Jul 01 21:05:59.536056 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:05:59.536015 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:04.536050 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:04.536011 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:09.535429 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:09.535383 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:09.535843 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:09.535496 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 21:06:14.535337 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:14.535288 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:17.029350 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:17.029243 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l"] Jul 01 21:06:17.029787 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:17.029495 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" containerID="cri-o://26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930" gracePeriod=30 Jul 01 21:06:19.535325 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:19.535289 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:19.620691 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:19.620648 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:24.535626 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:24.535587 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:24.620292 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:24.620238 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:27.301535 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.301503 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 21:06:27.503742 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.503646 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls\") pod \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " Jul 01 21:06:27.503742 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.503716 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-openshift-service-ca-bundle\") pod \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\" (UID: \"23cc18e6-9ac8-46b2-9b5e-c20702bbd904\") " Jul 01 21:06:27.504033 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.504010 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "23cc18e6-9ac8-46b2-9b5e-c20702bbd904" (UID: "23cc18e6-9ac8-46b2-9b5e-c20702bbd904"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:06:27.505784 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.505760 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "23cc18e6-9ac8-46b2-9b5e-c20702bbd904" (UID: "23cc18e6-9ac8-46b2-9b5e-c20702bbd904"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:06:27.605039 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.605005 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:06:27.605039 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.605035 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23cc18e6-9ac8-46b2-9b5e-c20702bbd904-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:06:27.664842 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.664803 2558 generic.go:356] "Generic (PLEG): container finished" podID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerID="ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59" exitCode=0 Jul 01 21:06:27.665000 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.664851 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" Jul 01 21:06:27.665000 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.664887 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" event={"ID":"23cc18e6-9ac8-46b2-9b5e-c20702bbd904","Type":"ContainerDied","Data":"ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59"} Jul 01 21:06:27.665000 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.664962 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4" event={"ID":"23cc18e6-9ac8-46b2-9b5e-c20702bbd904","Type":"ContainerDied","Data":"d3941b2e02c31bac5cd4a1477d65daa08b4b0495f76e151f676d9af29d90a49a"} Jul 01 21:06:27.665000 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.664985 2558 scope.go:117] "RemoveContainer" containerID="ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59" Jul 01 21:06:27.674167 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.674150 2558 scope.go:117] "RemoveContainer" containerID="ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59" Jul 01 21:06:27.674482 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:06:27.674460 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59\": container with ID starting with ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59 not found: ID does not exist" containerID="ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59" Jul 01 21:06:27.674574 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.674489 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59"} err="failed to get container status \"ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59\": rpc error: code = NotFound desc = could not find container \"ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59\": container with ID starting with ba3fda0fce977507a60a3e11e2e80ab2317102a03e83488bf69c69a396be6e59 not found: ID does not exist" Jul 01 21:06:27.685609 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.685580 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4"] Jul 01 21:06:27.689379 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.689353 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f347e-6f4bb9d8fd-f5wl4"] Jul 01 21:06:27.843497 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:27.843464 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" path="/var/lib/kubelet/pods/23cc18e6-9ac8-46b2-9b5e-c20702bbd904/volumes" Jul 01 21:06:29.620875 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:29.620836 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:29.621307 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:29.620932 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 21:06:34.620131 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:34.620088 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:39.620883 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:39.620845 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:44.620657 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:44.620618 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:06:47.177493 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.177458 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 21:06:47.255560 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.255526 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b90842f-4019-43ef-bc18-c15222f07ecc-proxy-tls\") pod \"0b90842f-4019-43ef-bc18-c15222f07ecc\" (UID: \"0b90842f-4019-43ef-bc18-c15222f07ecc\") " Jul 01 21:06:47.255747 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.255593 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90842f-4019-43ef-bc18-c15222f07ecc-openshift-service-ca-bundle\") pod \"0b90842f-4019-43ef-bc18-c15222f07ecc\" (UID: \"0b90842f-4019-43ef-bc18-c15222f07ecc\") " Jul 01 21:06:47.255973 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.255948 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b90842f-4019-43ef-bc18-c15222f07ecc-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "0b90842f-4019-43ef-bc18-c15222f07ecc" (UID: "0b90842f-4019-43ef-bc18-c15222f07ecc"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:06:47.257737 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.257710 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b90842f-4019-43ef-bc18-c15222f07ecc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0b90842f-4019-43ef-bc18-c15222f07ecc" (UID: "0b90842f-4019-43ef-bc18-c15222f07ecc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:06:47.356242 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.356211 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90842f-4019-43ef-bc18-c15222f07ecc-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:06:47.356242 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.356236 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b90842f-4019-43ef-bc18-c15222f07ecc-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:06:47.751058 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.750974 2558 generic.go:356] "Generic (PLEG): container finished" podID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerID="26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930" exitCode=0 Jul 01 21:06:47.751058 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.751033 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" Jul 01 21:06:47.751242 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.751036 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" event={"ID":"0b90842f-4019-43ef-bc18-c15222f07ecc","Type":"ContainerDied","Data":"26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930"} Jul 01 21:06:47.751242 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.751169 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l" event={"ID":"0b90842f-4019-43ef-bc18-c15222f07ecc","Type":"ContainerDied","Data":"38c17326791d03002ad08d3e25603db7e6084ea02f4d5a6b914ae29dc1b79e29"} Jul 01 21:06:47.751242 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.751190 2558 scope.go:117] "RemoveContainer" containerID="26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930" Jul 01 21:06:47.759332 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.759318 2558 scope.go:117] "RemoveContainer" containerID="26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930" Jul 01 21:06:47.759578 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:06:47.759561 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930\": container with ID starting with 26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930 not found: ID does not exist" containerID="26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930" Jul 01 21:06:47.759615 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.759585 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930"} err="failed to get container status \"26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930\": rpc error: code = NotFound desc = could not find container \"26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930\": container with ID starting with 26b6d530e94b2a2002f77807368672c7b9438e073d28b922ba2a35030bc89930 not found: ID does not exist" Jul 01 21:06:47.770999 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.770978 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l"] Jul 01 21:06:47.773991 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.773970 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a93e1-84bdb6d745-qlk2l"] Jul 01 21:06:47.843123 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:47.843096 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" path="/var/lib/kubelet/pods/0b90842f-4019-43ef-bc18-c15222f07ecc/volumes" Jul 01 21:06:55.863827 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:55.863801 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:06:55.867311 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:06:55.867291 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:07:07.373475 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.373443 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf"] Jul 01 21:07:07.373870 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.373754 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" Jul 01 21:07:07.373870 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.373766 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" Jul 01 21:07:07.373870 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.373779 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" Jul 01 21:07:07.373870 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.373784 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" Jul 01 21:07:07.373870 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.373827 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b90842f-4019-43ef-bc18-c15222f07ecc" containerName="sequence-graph-a93e1" Jul 01 21:07:07.373870 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.373836 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="23cc18e6-9ac8-46b2-9b5e-c20702bbd904" containerName="switch-graph-f347e" Jul 01 21:07:07.378320 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.378248 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:07.380242 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.380220 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-f1d61-kube-rbac-proxy-sar-config\"" Jul 01 21:07:07.380330 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.380239 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vwmkn\"" Jul 01 21:07:07.380845 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.380829 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-f1d61-serving-cert\"" Jul 01 21:07:07.380898 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.380851 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 21:07:07.386400 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.386379 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf"] Jul 01 21:07:07.519229 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.519199 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls\") pod \"ensemble-graph-f1d61-6f568488db-8x8cf\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:07.519229 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.519233 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-openshift-service-ca-bundle\") pod \"ensemble-graph-f1d61-6f568488db-8x8cf\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:07.619700 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.619660 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls\") pod \"ensemble-graph-f1d61-6f568488db-8x8cf\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:07.619700 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.619702 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-openshift-service-ca-bundle\") pod \"ensemble-graph-f1d61-6f568488db-8x8cf\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:07.619917 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:07:07.619815 2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-f1d61-serving-cert: secret "ensemble-graph-f1d61-serving-cert" not found Jul 01 21:07:07.619917 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:07:07.619891 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls podName:d8d93fb8-5a12-4e49-9b36-83a84dd0865d nodeName:}" failed. No retries permitted until 2026-07-01 21:07:08.119873814 +0000 UTC m=+1212.728316668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls") pod "ensemble-graph-f1d61-6f568488db-8x8cf" (UID: "d8d93fb8-5a12-4e49-9b36-83a84dd0865d") : secret "ensemble-graph-f1d61-serving-cert" not found Jul 01 21:07:07.620299 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:07.620252 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-openshift-service-ca-bundle\") pod \"ensemble-graph-f1d61-6f568488db-8x8cf\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:08.124488 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:08.124450 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls\") pod \"ensemble-graph-f1d61-6f568488db-8x8cf\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:08.126962 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:08.126944 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls\") pod \"ensemble-graph-f1d61-6f568488db-8x8cf\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:08.289587 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:08.289544 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:08.422739 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:08.422715 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf"] Jul 01 21:07:08.424789 ip-10-0-130-150 kubenswrapper[2558]: W0701 21:07:08.424762 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d93fb8_5a12_4e49_9b36_83a84dd0865d.slice/crio-f121ac79db6576dffb43888f95fd2551086dd1ead8deed2bd1ce3bd49de45f93 WatchSource:0}: Error finding container f121ac79db6576dffb43888f95fd2551086dd1ead8deed2bd1ce3bd49de45f93: Status 404 returned error can't find the container with id f121ac79db6576dffb43888f95fd2551086dd1ead8deed2bd1ce3bd49de45f93 Jul 01 21:07:08.847568 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:08.847533 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" event={"ID":"d8d93fb8-5a12-4e49-9b36-83a84dd0865d","Type":"ContainerStarted","Data":"2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff"} Jul 01 21:07:08.847746 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:08.847588 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" event={"ID":"d8d93fb8-5a12-4e49-9b36-83a84dd0865d","Type":"ContainerStarted","Data":"f121ac79db6576dffb43888f95fd2551086dd1ead8deed2bd1ce3bd49de45f93"} Jul 01 21:07:08.847746 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:08.847640 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:08.864381 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:08.864326 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" podStartSLOduration=1.864308268 podStartE2EDuration="1.864308268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:07:08.862187645 +0000 UTC m=+1213.470630519" watchObservedRunningTime="2026-07-01 21:07:08.864308268 +0000 UTC m=+1213.472751143" Jul 01 21:07:14.857646 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:14.857613 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:17.440679 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:17.440649 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf"] Jul 01 21:07:17.441087 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:17.440824 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" containerID="cri-o://2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff" gracePeriod=30 Jul 01 21:07:19.855696 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:19.855657 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:24.856407 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:24.856368 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:27.218009 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.217971 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm"] Jul 01 21:07:27.221633 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.221612 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:27.223827 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.223796 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-7cf3f-kube-rbac-proxy-sar-config\"" Jul 01 21:07:27.223954 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.223843 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-7cf3f-serving-cert\"" Jul 01 21:07:27.228236 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.228213 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm"] Jul 01 21:07:27.279247 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.279212 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45078883-6fc1-47bf-8657-f28d04d493f6-openshift-service-ca-bundle\") pod \"sequence-graph-7cf3f-6ddb7696d5-hzlbm\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:27.279423 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.279275 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls\") pod \"sequence-graph-7cf3f-6ddb7696d5-hzlbm\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:27.380537 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.380502 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45078883-6fc1-47bf-8657-f28d04d493f6-openshift-service-ca-bundle\") pod \"sequence-graph-7cf3f-6ddb7696d5-hzlbm\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:27.380691 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.380554 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls\") pod \"sequence-graph-7cf3f-6ddb7696d5-hzlbm\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:27.380691 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:07:27.380655 2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-7cf3f-serving-cert: secret "sequence-graph-7cf3f-serving-cert" not found Jul 01 21:07:27.380779 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:07:27.380728 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls podName:45078883-6fc1-47bf-8657-f28d04d493f6 nodeName:}" failed. No retries permitted until 2026-07-01 21:07:27.880705404 +0000 UTC m=+1232.489148258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls") pod "sequence-graph-7cf3f-6ddb7696d5-hzlbm" (UID: "45078883-6fc1-47bf-8657-f28d04d493f6") : secret "sequence-graph-7cf3f-serving-cert" not found Jul 01 21:07:27.381185 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.381167 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45078883-6fc1-47bf-8657-f28d04d493f6-openshift-service-ca-bundle\") pod \"sequence-graph-7cf3f-6ddb7696d5-hzlbm\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:27.883637 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.883606 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls\") pod \"sequence-graph-7cf3f-6ddb7696d5-hzlbm\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:27.886218 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:27.886190 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls\") pod \"sequence-graph-7cf3f-6ddb7696d5-hzlbm\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:28.132815 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:28.132781 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:28.259296 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:28.259251 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm"] Jul 01 21:07:28.261293 ip-10-0-130-150 kubenswrapper[2558]: W0701 21:07:28.261269 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45078883_6fc1_47bf_8657_f28d04d493f6.slice/crio-6ab98f9bd14e54a0d73e09d8bf9cd9865fc77293c5a6c165244cf06d4d9ecca3 WatchSource:0}: Error finding container 6ab98f9bd14e54a0d73e09d8bf9cd9865fc77293c5a6c165244cf06d4d9ecca3: Status 404 returned error can't find the container with id 6ab98f9bd14e54a0d73e09d8bf9cd9865fc77293c5a6c165244cf06d4d9ecca3 Jul 01 21:07:28.933176 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:28.933138 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" event={"ID":"45078883-6fc1-47bf-8657-f28d04d493f6","Type":"ContainerStarted","Data":"53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce"} Jul 01 21:07:28.933382 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:28.933192 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" event={"ID":"45078883-6fc1-47bf-8657-f28d04d493f6","Type":"ContainerStarted","Data":"6ab98f9bd14e54a0d73e09d8bf9cd9865fc77293c5a6c165244cf06d4d9ecca3"} Jul 01 21:07:28.933382 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:28.933240 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:28.949227 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:28.949177 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" podStartSLOduration=1.949163087 podStartE2EDuration="1.949163087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:07:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:07:28.947626237 +0000 UTC m=+1233.556069112" watchObservedRunningTime="2026-07-01 21:07:28.949163087 +0000 UTC m=+1233.557605962" Jul 01 21:07:29.855591 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:29.855548 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:29.856015 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:29.855640 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:34.855914 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:34.855873 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:34.948698 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:34.948670 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:37.300135 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:37.300099 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm"] Jul 01 21:07:37.300671 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:37.300343 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" containerID="cri-o://53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce" gracePeriod=30 Jul 01 21:07:39.856235 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:39.856195 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:39.946742 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:39.946704 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:44.855637 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:44.855597 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:44.946646 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:44.946609 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:47.593751 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:47.593725 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:47.637218 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:47.637183 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls\") pod \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " Jul 01 21:07:47.637218 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:47.637222 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-openshift-service-ca-bundle\") pod \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\" (UID: \"d8d93fb8-5a12-4e49-9b36-83a84dd0865d\") " Jul 01 21:07:47.637653 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:47.637628 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d8d93fb8-5a12-4e49-9b36-83a84dd0865d" (UID: "d8d93fb8-5a12-4e49-9b36-83a84dd0865d"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:07:47.639410 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:47.639387 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d8d93fb8-5a12-4e49-9b36-83a84dd0865d" (UID: "d8d93fb8-5a12-4e49-9b36-83a84dd0865d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:07:47.738595 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:47.738506 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:07:47.738595 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:47.738535 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8d93fb8-5a12-4e49-9b36-83a84dd0865d-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:07:48.019551 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.019454 2558 generic.go:356] "Generic (PLEG): container finished" podID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerID="2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff" exitCode=0 Jul 01 21:07:48.019551 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.019496 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" Jul 01 21:07:48.019551 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.019538 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" event={"ID":"d8d93fb8-5a12-4e49-9b36-83a84dd0865d","Type":"ContainerDied","Data":"2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff"} Jul 01 21:07:48.019801 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.019585 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf" event={"ID":"d8d93fb8-5a12-4e49-9b36-83a84dd0865d","Type":"ContainerDied","Data":"f121ac79db6576dffb43888f95fd2551086dd1ead8deed2bd1ce3bd49de45f93"} Jul 01 21:07:48.019801 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.019607 2558 scope.go:117] "RemoveContainer" containerID="2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff" Jul 01 21:07:48.027804 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.027774 2558 scope.go:117] "RemoveContainer" containerID="2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff" Jul 01 21:07:48.028059 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:07:48.028038 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff\": container with ID starting with 2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff not found: ID does not exist" containerID="2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff" Jul 01 21:07:48.028099 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.028069 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff"} err="failed to get container status \"2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff\": rpc error: code = NotFound desc = could not find container \"2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff\": container with ID starting with 2aaafbe82bb9cdebcbf9ac8a4560f4c8a1760bbb39f6fd095e6cac1507eb5dff not found: ID does not exist" Jul 01 21:07:48.036771 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.036725 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf"] Jul 01 21:07:48.039346 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:48.039327 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-f1d61-6f568488db-8x8cf"] Jul 01 21:07:49.844057 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:49.844013 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" path="/var/lib/kubelet/pods/d8d93fb8-5a12-4e49-9b36-83a84dd0865d/volumes" Jul 01 21:07:49.946751 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:49.946713 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:49.946916 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:49.946814 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:07:54.946844 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:54.946807 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:07:59.946447 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:07:59.946410 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:08:04.946466 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:04.946429 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:08:07.945978 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:07.945954 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:08:08.101393 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.101307 2558 generic.go:356] "Generic (PLEG): container finished" podID="45078883-6fc1-47bf-8657-f28d04d493f6" containerID="53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce" exitCode=0 Jul 01 21:08:08.101393 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.101354 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" Jul 01 21:08:08.101562 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.101387 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" event={"ID":"45078883-6fc1-47bf-8657-f28d04d493f6","Type":"ContainerDied","Data":"53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce"} Jul 01 21:08:08.101562 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.101443 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm" event={"ID":"45078883-6fc1-47bf-8657-f28d04d493f6","Type":"ContainerDied","Data":"6ab98f9bd14e54a0d73e09d8bf9cd9865fc77293c5a6c165244cf06d4d9ecca3"} Jul 01 21:08:08.101562 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.101473 2558 scope.go:117] "RemoveContainer" containerID="53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce" Jul 01 21:08:08.103667 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.103630 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls\") pod \"45078883-6fc1-47bf-8657-f28d04d493f6\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " Jul 01 21:08:08.103779 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.103676 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45078883-6fc1-47bf-8657-f28d04d493f6-openshift-service-ca-bundle\") pod \"45078883-6fc1-47bf-8657-f28d04d493f6\" (UID: \"45078883-6fc1-47bf-8657-f28d04d493f6\") " Jul 01 21:08:08.104026 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.104004 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45078883-6fc1-47bf-8657-f28d04d493f6-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "45078883-6fc1-47bf-8657-f28d04d493f6" (UID: "45078883-6fc1-47bf-8657-f28d04d493f6"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:08:08.105705 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.105687 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "45078883-6fc1-47bf-8657-f28d04d493f6" (UID: "45078883-6fc1-47bf-8657-f28d04d493f6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:08:08.115057 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.115035 2558 scope.go:117] "RemoveContainer" containerID="53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce" Jul 01 21:08:08.115417 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:08:08.115395 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce\": container with ID starting with 53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce not found: ID does not exist" containerID="53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce" Jul 01 21:08:08.115461 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.115430 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce"} err="failed to get container status \"53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce\": rpc error: code = NotFound desc = could not find container \"53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce\": container with ID starting with 53f7ff1d774b2d073f7e9ce20bbe160e4cee1e91a20799025515c531e5eb4cce not found: ID does not exist" Jul 01 21:08:08.205099 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.205066 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/45078883-6fc1-47bf-8657-f28d04d493f6-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:08:08.205099 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.205093 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45078883-6fc1-47bf-8657-f28d04d493f6-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:08:08.421755 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.421721 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm"] Jul 01 21:08:08.425410 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:08.425387 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7cf3f-6ddb7696d5-hzlbm"] Jul 01 21:08:09.843618 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:09.843582 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" path="/var/lib/kubelet/pods/45078883-6fc1-47bf-8657-f28d04d493f6/volumes" Jul 01 21:08:17.650938 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.650905 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7"] Jul 01 21:08:17.651316 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.651197 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" Jul 01 21:08:17.651316 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.651207 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" Jul 01 21:08:17.651316 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.651224 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" Jul 01 21:08:17.651316 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.651230 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" Jul 01 21:08:17.651316 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.651304 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="45078883-6fc1-47bf-8657-f28d04d493f6" containerName="sequence-graph-7cf3f" Jul 01 21:08:17.651316 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.651312 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8d93fb8-5a12-4e49-9b36-83a84dd0865d" containerName="ensemble-graph-f1d61" Jul 01 21:08:17.655560 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.655544 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:17.657814 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.657786 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-86dc6-serving-cert\"" Jul 01 21:08:17.657814 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.657800 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-86dc6-kube-rbac-proxy-sar-config\"" Jul 01 21:08:17.657814 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.657789 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vwmkn\"" Jul 01 21:08:17.658097 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.657804 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 21:08:17.660639 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.660597 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7"] Jul 01 21:08:17.780467 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.780431 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-openshift-service-ca-bundle\") pod \"ensemble-graph-86dc6-9854ff79c-rrpx7\" (UID: \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\") " pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:17.780677 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.780502 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-proxy-tls\") pod \"ensemble-graph-86dc6-9854ff79c-rrpx7\" (UID: \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\") " pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:17.881721 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.881691 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-proxy-tls\") pod \"ensemble-graph-86dc6-9854ff79c-rrpx7\" (UID: \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\") " pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:17.881861 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.881730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-openshift-service-ca-bundle\") pod \"ensemble-graph-86dc6-9854ff79c-rrpx7\" (UID: \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\") " pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:17.882316 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.882300 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-openshift-service-ca-bundle\") pod \"ensemble-graph-86dc6-9854ff79c-rrpx7\" (UID: \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\") " pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:17.884157 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.884139 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-proxy-tls\") pod \"ensemble-graph-86dc6-9854ff79c-rrpx7\" (UID: \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\") " pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:17.966988 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:17.966909 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:18.094876 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:18.094853 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7"] Jul 01 21:08:18.097090 ip-10-0-130-150 kubenswrapper[2558]: W0701 21:08:18.097065 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95fe24d_64da_4e14_9b30_c39ee5ba8f12.slice/crio-2c93d700cb8100959af5388b94171b6b29ea9e0dceceed6ef986a02aaf1956dc WatchSource:0}: Error finding container 2c93d700cb8100959af5388b94171b6b29ea9e0dceceed6ef986a02aaf1956dc: Status 404 returned error can't find the container with id 2c93d700cb8100959af5388b94171b6b29ea9e0dceceed6ef986a02aaf1956dc Jul 01 21:08:18.149771 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:18.149739 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" event={"ID":"a95fe24d-64da-4e14-9b30-c39ee5ba8f12","Type":"ContainerStarted","Data":"2c93d700cb8100959af5388b94171b6b29ea9e0dceceed6ef986a02aaf1956dc"} Jul 01 21:08:19.155770 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:19.155733 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" event={"ID":"a95fe24d-64da-4e14-9b30-c39ee5ba8f12","Type":"ContainerStarted","Data":"0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c"} Jul 01 21:08:19.156232 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:19.155875 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:19.172728 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:19.172685 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" podStartSLOduration=2.172672606 podStartE2EDuration="2.172672606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:08:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:08:19.171121403 +0000 UTC m=+1283.779564280" watchObservedRunningTime="2026-07-01 21:08:19.172672606 +0000 UTC m=+1283.781115480" Jul 01 21:08:25.165756 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:25.165726 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:08:37.537029 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.537000 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26"] Jul 01 21:08:37.540882 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.540858 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:37.543043 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.543018 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c5814-kube-rbac-proxy-sar-config\"" Jul 01 21:08:37.543229 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.543025 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c5814-serving-cert\"" Jul 01 21:08:37.548849 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.548523 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26"] Jul 01 21:08:37.643903 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.643866 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-proxy-tls\") pod \"sequence-graph-c5814-8549fd89d-pwg26\" (UID: \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\") " pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:37.644073 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.643973 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-openshift-service-ca-bundle\") pod \"sequence-graph-c5814-8549fd89d-pwg26\" (UID: \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\") " pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:37.745364 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.745330 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-proxy-tls\") pod \"sequence-graph-c5814-8549fd89d-pwg26\" (UID: \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\") " pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:37.745573 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.745424 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-openshift-service-ca-bundle\") pod \"sequence-graph-c5814-8549fd89d-pwg26\" (UID: \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\") " pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:37.746029 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.746006 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-openshift-service-ca-bundle\") pod \"sequence-graph-c5814-8549fd89d-pwg26\" (UID: \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\") " pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:37.747907 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.747884 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-proxy-tls\") pod \"sequence-graph-c5814-8549fd89d-pwg26\" (UID: \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\") " pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:37.854355 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.854313 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:37.988467 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:37.988442 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26"] Jul 01 21:08:37.990473 ip-10-0-130-150 kubenswrapper[2558]: W0701 21:08:37.990436 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfe64ae_33f2_40a0_b9f8_b81faf35a0ac.slice/crio-29b6014dc5efbcf15cfa5eebf8910a9dfd7ac713c2541ae3dc61f447c269b768 WatchSource:0}: Error finding container 29b6014dc5efbcf15cfa5eebf8910a9dfd7ac713c2541ae3dc61f447c269b768: Status 404 returned error can't find the container with id 29b6014dc5efbcf15cfa5eebf8910a9dfd7ac713c2541ae3dc61f447c269b768 Jul 01 21:08:38.237414 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:38.237322 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" event={"ID":"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac","Type":"ContainerStarted","Data":"bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6"} Jul 01 21:08:38.237414 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:38.237371 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" event={"ID":"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac","Type":"ContainerStarted","Data":"29b6014dc5efbcf15cfa5eebf8910a9dfd7ac713c2541ae3dc61f447c269b768"} Jul 01 21:08:38.237414 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:38.237411 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:08:38.254087 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:38.254028 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" podStartSLOduration=1.254011886 podStartE2EDuration="1.254011886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:08:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:08:38.25253364 +0000 UTC m=+1302.860976517" watchObservedRunningTime="2026-07-01 21:08:38.254011886 +0000 UTC m=+1302.862454761" Jul 01 21:08:44.246974 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:08:44.246946 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:11:55.887543 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:11:55.887505 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:11:55.891594 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:11:55.891574 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:16:32.397402 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:32.397364 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7"] Jul 01 21:16:32.400000 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:32.397619 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" containerID="cri-o://0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c" gracePeriod=30 Jul 01 21:16:35.163975 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:35.163926 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:16:40.164318 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:40.164284 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:16:45.164442 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:45.164390 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:16:45.164885 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:45.164517 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:16:50.164633 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:50.164550 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:16:52.148000 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:52.147963 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26"] Jul 01 21:16:52.148382 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:52.148207 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" containerID="cri-o://bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6" gracePeriod=30 Jul 01 21:16:54.245980 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:54.245932 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:16:55.164403 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:55.164359 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:16:55.910810 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:55.910782 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:16:55.916468 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:55.916447 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:16:59.246085 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:16:59.246040 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:17:00.163854 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:00.163818 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:17:02.546968 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:02.546944 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:17:02.610371 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:02.610336 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-proxy-tls\") pod \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\" (UID: \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\") " Jul 01 21:17:02.610539 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:02.610397 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-openshift-service-ca-bundle\") pod \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\" (UID: \"a95fe24d-64da-4e14-9b30-c39ee5ba8f12\") " Jul 01 21:17:02.610780 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:02.610751 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "a95fe24d-64da-4e14-9b30-c39ee5ba8f12" (UID: "a95fe24d-64da-4e14-9b30-c39ee5ba8f12"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:17:02.612586 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:02.612558 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a95fe24d-64da-4e14-9b30-c39ee5ba8f12" (UID: "a95fe24d-64da-4e14-9b30-c39ee5ba8f12"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:17:02.711062 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:02.710981 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:17:02.711062 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:02.711012 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a95fe24d-64da-4e14-9b30-c39ee5ba8f12-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:17:03.320163 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.320126 2558 generic.go:356] "Generic (PLEG): container finished" podID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerID="0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c" exitCode=0 Jul 01 21:17:03.320471 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.320170 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" Jul 01 21:17:03.320471 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.320181 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" event={"ID":"a95fe24d-64da-4e14-9b30-c39ee5ba8f12","Type":"ContainerDied","Data":"0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c"} Jul 01 21:17:03.320471 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.320230 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7" event={"ID":"a95fe24d-64da-4e14-9b30-c39ee5ba8f12","Type":"ContainerDied","Data":"2c93d700cb8100959af5388b94171b6b29ea9e0dceceed6ef986a02aaf1956dc"} Jul 01 21:17:03.320471 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.320252 2558 scope.go:117] "RemoveContainer" containerID="0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c" Jul 01 21:17:03.332350 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.332326 2558 scope.go:117] "RemoveContainer" containerID="0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c" Jul 01 21:17:03.332707 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:17:03.332677 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c\": container with ID starting with 0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c not found: ID does not exist" containerID="0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c" Jul 01 21:17:03.332799 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.332717 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c"} err="failed to get container status \"0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c\": rpc error: code = NotFound desc = could not find container \"0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c\": container with ID starting with 0796afc4a6e050a4b8be7bd877a5319b799b1777357156185286366a63fe912c not found: ID does not exist" Jul 01 21:17:03.343410 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.343382 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7"] Jul 01 21:17:03.348861 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.348841 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-86dc6-9854ff79c-rrpx7"] Jul 01 21:17:03.843032 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:03.842998 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" path="/var/lib/kubelet/pods/a95fe24d-64da-4e14-9b30-c39ee5ba8f12/volumes" Jul 01 21:17:04.245952 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:04.245868 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:17:04.246092 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:04.245969 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:17:09.245184 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:09.245145 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:17:14.245061 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:14.245020 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:17:19.245363 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:19.245320 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:17:22.324592 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.324565 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:17:22.355557 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.355522 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-proxy-tls\") pod \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\" (UID: \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\") " Jul 01 21:17:22.355689 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.355581 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-openshift-service-ca-bundle\") pod \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\" (UID: \"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac\") " Jul 01 21:17:22.355947 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.355917 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" (UID: "5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:17:22.357683 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.357661 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" (UID: "5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:17:22.400358 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.400322 2558 generic.go:356] "Generic (PLEG): container finished" podID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerID="bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6" exitCode=0 Jul 01 21:17:22.400486 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.400374 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" Jul 01 21:17:22.400486 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.400405 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" event={"ID":"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac","Type":"ContainerDied","Data":"bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6"} Jul 01 21:17:22.400486 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.400462 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26" event={"ID":"5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac","Type":"ContainerDied","Data":"29b6014dc5efbcf15cfa5eebf8910a9dfd7ac713c2541ae3dc61f447c269b768"} Jul 01 21:17:22.400577 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.400486 2558 scope.go:117] "RemoveContainer" containerID="bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6" Jul 01 21:17:22.408837 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.408822 2558 scope.go:117] "RemoveContainer" containerID="bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6" Jul 01 21:17:22.409095 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:17:22.409077 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6\": container with ID starting with bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6 not found: ID does not exist" containerID="bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6" Jul 01 21:17:22.409136 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.409103 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6"} err="failed to get container status \"bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6\": rpc error: code = NotFound desc = could not find container \"bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6\": container with ID starting with bfb7f9beb6ac742f7debe60867bbd9c51169b697534fa9b31b74d6177e5543c6 not found: ID does not exist" Jul 01 21:17:22.421628 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.421602 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26"] Jul 01 21:17:22.423023 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.423004 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c5814-8549fd89d-pwg26"] Jul 01 21:17:22.456177 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.456143 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:17:22.456177 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:22.456170 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:17:23.843691 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:23.843664 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" path="/var/lib/kubelet/pods/5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac/volumes" Jul 01 21:17:42.646800 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.646722 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4"] Jul 01 21:17:42.647171 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.647044 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" Jul 01 21:17:42.647171 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.647056 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" Jul 01 21:17:42.647171 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.647069 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" Jul 01 21:17:42.647171 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.647078 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" Jul 01 21:17:42.647171 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.647157 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bfe64ae-33f2-40a0-b9f8-b81faf35a0ac" containerName="sequence-graph-c5814" Jul 01 21:17:42.647171 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.647168 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a95fe24d-64da-4e14-9b30-c39ee5ba8f12" containerName="ensemble-graph-86dc6" Jul 01 21:17:42.650100 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.650084 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:42.658599 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.658566 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-3b58c-kube-rbac-proxy-sar-config\"" Jul 01 21:17:42.658855 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.658837 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-vwmkn\"" Jul 01 21:17:42.659135 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.659119 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 21:17:42.659458 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.659442 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-3b58c-serving-cert\"" Jul 01 21:17:42.679455 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.679429 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4"] Jul 01 21:17:42.709651 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.709628 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-openshift-service-ca-bundle\") pod \"splitter-graph-3b58c-b7d74687d-t4jh4\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:42.709762 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.709702 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls\") pod \"splitter-graph-3b58c-b7d74687d-t4jh4\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:42.810197 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.810159 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-openshift-service-ca-bundle\") pod \"splitter-graph-3b58c-b7d74687d-t4jh4\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:42.810401 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.810230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls\") pod \"splitter-graph-3b58c-b7d74687d-t4jh4\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:42.810401 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:17:42.810344 2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-3b58c-serving-cert: secret "splitter-graph-3b58c-serving-cert" not found Jul 01 21:17:42.810401 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:17:42.810402 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls podName:d6a8f4c6-95e2-4d80-a91f-b878dd968cf8 nodeName:}" failed. No retries permitted until 2026-07-01 21:17:43.310387502 +0000 UTC m=+1847.918830354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls") pod "splitter-graph-3b58c-b7d74687d-t4jh4" (UID: "d6a8f4c6-95e2-4d80-a91f-b878dd968cf8") : secret "splitter-graph-3b58c-serving-cert" not found Jul 01 21:17:42.810876 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:42.810850 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-openshift-service-ca-bundle\") pod \"splitter-graph-3b58c-b7d74687d-t4jh4\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:43.314427 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:43.314381 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls\") pod \"splitter-graph-3b58c-b7d74687d-t4jh4\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:43.316927 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:43.316903 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls\") pod \"splitter-graph-3b58c-b7d74687d-t4jh4\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:43.560374 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:43.560339 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:43.689127 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:43.689104 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4"] Jul 01 21:17:43.691317 ip-10-0-130-150 kubenswrapper[2558]: W0701 21:17:43.691289 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a8f4c6_95e2_4d80_a91f_b878dd968cf8.slice/crio-11639dcb9f6b145dd2b1ddb843c8e4e747a08cfc917bab43c67172c1f2b86760 WatchSource:0}: Error finding container 11639dcb9f6b145dd2b1ddb843c8e4e747a08cfc917bab43c67172c1f2b86760: Status 404 returned error can't find the container with id 11639dcb9f6b145dd2b1ddb843c8e4e747a08cfc917bab43c67172c1f2b86760 Jul 01 21:17:44.496456 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:44.496415 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" event={"ID":"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8","Type":"ContainerStarted","Data":"bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67"} Jul 01 21:17:44.496665 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:44.496468 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" event={"ID":"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8","Type":"ContainerStarted","Data":"11639dcb9f6b145dd2b1ddb843c8e4e747a08cfc917bab43c67172c1f2b86760"} Jul 01 21:17:44.496665 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:44.496493 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:44.514465 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:44.514427 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" podStartSLOduration=2.514413626 podStartE2EDuration="2.514413626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:17:44.513559204 +0000 UTC m=+1849.122002078" watchObservedRunningTime="2026-07-01 21:17:44.514413626 +0000 UTC m=+1849.122856525" Jul 01 21:17:50.505702 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:50.505671 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:17:52.708134 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:52.708102 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4"] Jul 01 21:17:52.708538 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:52.708298 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" containerID="cri-o://bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67" gracePeriod=30 Jul 01 21:17:55.504148 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:17:55.504108 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:18:00.504644 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:00.504601 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:18:02.367301 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.367248 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt"] Jul 01 21:18:02.370516 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.370500 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:02.372676 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.372643 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-ce993-kube-rbac-proxy-sar-config\"" Jul 01 21:18:02.372794 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.372754 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-ce993-serving-cert\"" Jul 01 21:18:02.382146 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.382123 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt"] Jul 01 21:18:02.481165 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.481123 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f754b68d-c342-4b58-a248-c68fbc970586-openshift-service-ca-bundle\") pod \"switch-graph-ce993-845959cd86-5gstt\" (UID: \"f754b68d-c342-4b58-a248-c68fbc970586\") " pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:02.481372 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.481176 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f754b68d-c342-4b58-a248-c68fbc970586-proxy-tls\") pod \"switch-graph-ce993-845959cd86-5gstt\" (UID: \"f754b68d-c342-4b58-a248-c68fbc970586\") " pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:02.582213 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.582178 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f754b68d-c342-4b58-a248-c68fbc970586-openshift-service-ca-bundle\") pod \"switch-graph-ce993-845959cd86-5gstt\" (UID: \"f754b68d-c342-4b58-a248-c68fbc970586\") " pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:02.582213 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.582219 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f754b68d-c342-4b58-a248-c68fbc970586-proxy-tls\") pod \"switch-graph-ce993-845959cd86-5gstt\" (UID: \"f754b68d-c342-4b58-a248-c68fbc970586\") " pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:02.582885 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.582857 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f754b68d-c342-4b58-a248-c68fbc970586-openshift-service-ca-bundle\") pod \"switch-graph-ce993-845959cd86-5gstt\" (UID: \"f754b68d-c342-4b58-a248-c68fbc970586\") " pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:02.584734 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.584714 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f754b68d-c342-4b58-a248-c68fbc970586-proxy-tls\") pod \"switch-graph-ce993-845959cd86-5gstt\" (UID: \"f754b68d-c342-4b58-a248-c68fbc970586\") " pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:02.680500 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.680395 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:02.815111 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:02.815082 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt"] Jul 01 21:18:02.817558 ip-10-0-130-150 kubenswrapper[2558]: W0701 21:18:02.817532 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf754b68d_c342_4b58_a248_c68fbc970586.slice/crio-9580379ef215816ad47bcb0bc884332cdafa7ab0db5366d6b4a1b482f42d1c57 WatchSource:0}: Error finding container 9580379ef215816ad47bcb0bc884332cdafa7ab0db5366d6b4a1b482f42d1c57: Status 404 returned error can't find the container with id 9580379ef215816ad47bcb0bc884332cdafa7ab0db5366d6b4a1b482f42d1c57 Jul 01 21:18:03.574022 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:03.573984 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" event={"ID":"f754b68d-c342-4b58-a248-c68fbc970586","Type":"ContainerStarted","Data":"b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb"} Jul 01 21:18:03.574443 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:03.574046 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" event={"ID":"f754b68d-c342-4b58-a248-c68fbc970586","Type":"ContainerStarted","Data":"9580379ef215816ad47bcb0bc884332cdafa7ab0db5366d6b4a1b482f42d1c57"} Jul 01 21:18:03.574443 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:03.574090 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:03.592668 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:03.592617 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" podStartSLOduration=1.592603443 podStartE2EDuration="1.592603443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:18:03.590977088 +0000 UTC m=+1868.199419963" watchObservedRunningTime="2026-07-01 21:18:03.592603443 +0000 UTC m=+1868.201046355" Jul 01 21:18:05.504065 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:05.504027 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:18:05.504555 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:05.504140 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:18:09.584730 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:09.584701 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:18:10.504461 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:10.504422 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:18:15.504067 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:15.504031 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:18:20.505053 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:20.505017 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:18:22.858946 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:22.858922 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:18:22.932752 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:22.932723 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-openshift-service-ca-bundle\") pod \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " Jul 01 21:18:22.932916 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:22.932832 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls\") pod \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\" (UID: \"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8\") " Jul 01 21:18:22.933099 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:22.933071 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" (UID: "d6a8f4c6-95e2-4d80-a91f-b878dd968cf8"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:18:22.934876 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:22.934853 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" (UID: "d6a8f4c6-95e2-4d80-a91f-b878dd968cf8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:18:23.034437 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.034363 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:18:23.034437 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.034389 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:18:23.654373 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.654338 2558 generic.go:356] "Generic (PLEG): container finished" podID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerID="bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67" exitCode=0 Jul 01 21:18:23.654541 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.654391 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" Jul 01 21:18:23.654541 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.654428 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" event={"ID":"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8","Type":"ContainerDied","Data":"bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67"} Jul 01 21:18:23.654541 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.654488 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4" event={"ID":"d6a8f4c6-95e2-4d80-a91f-b878dd968cf8","Type":"ContainerDied","Data":"11639dcb9f6b145dd2b1ddb843c8e4e747a08cfc917bab43c67172c1f2b86760"} Jul 01 21:18:23.654541 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.654515 2558 scope.go:117] "RemoveContainer" containerID="bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67" Jul 01 21:18:23.663073 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.663054 2558 scope.go:117] "RemoveContainer" containerID="bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67" Jul 01 21:18:23.663376 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:18:23.663355 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67\": container with ID starting with bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67 not found: ID does not exist" containerID="bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67" Jul 01 21:18:23.663440 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.663385 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67"} err="failed to get container status \"bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67\": rpc error: code = NotFound desc = could not find container \"bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67\": container with ID starting with bdf8285b51fed2178de6a878932aa0c65e46aace31b14b7a0fe4abf3a64f6a67 not found: ID does not exist" Jul 01 21:18:23.676273 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.676236 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4"] Jul 01 21:18:23.680885 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.680868 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-3b58c-b7d74687d-t4jh4"] Jul 01 21:18:23.843219 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:23.843172 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" path="/var/lib/kubelet/pods/d6a8f4c6-95e2-4d80-a91f-b878dd968cf8/volumes" Jul 01 21:18:52.929545 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:52.929508 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst"] Jul 01 21:18:52.930009 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:52.929830 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" Jul 01 21:18:52.930009 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:52.929841 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" Jul 01 21:18:52.930009 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:52.929892 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6a8f4c6-95e2-4d80-a91f-b878dd968cf8" containerName="splitter-graph-3b58c" Jul 01 21:18:52.933183 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:52.933161 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:52.936959 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:52.936929 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-6396f-kube-rbac-proxy-sar-config\"" Jul 01 21:18:52.937084 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:52.937072 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-6396f-serving-cert\"" Jul 01 21:18:52.944802 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:52.944773 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst"] Jul 01 21:18:53.072606 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.072574 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785dc6c-a28a-43a4-97f9-904212448f0e-openshift-service-ca-bundle\") pod \"splitter-graph-6396f-548474fcc8-mxqst\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:53.072773 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.072616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls\") pod \"splitter-graph-6396f-548474fcc8-mxqst\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:53.173116 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.173079 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785dc6c-a28a-43a4-97f9-904212448f0e-openshift-service-ca-bundle\") pod \"splitter-graph-6396f-548474fcc8-mxqst\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:53.173299 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.173129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls\") pod \"splitter-graph-6396f-548474fcc8-mxqst\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:53.173299 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:18:53.173251 2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-6396f-serving-cert: secret "splitter-graph-6396f-serving-cert" not found Jul 01 21:18:53.173377 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:18:53.173340 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls podName:8785dc6c-a28a-43a4-97f9-904212448f0e nodeName:}" failed. No retries permitted until 2026-07-01 21:18:53.673323126 +0000 UTC m=+1918.281765978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls") pod "splitter-graph-6396f-548474fcc8-mxqst" (UID: "8785dc6c-a28a-43a4-97f9-904212448f0e") : secret "splitter-graph-6396f-serving-cert" not found Jul 01 21:18:53.173748 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.173726 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785dc6c-a28a-43a4-97f9-904212448f0e-openshift-service-ca-bundle\") pod \"splitter-graph-6396f-548474fcc8-mxqst\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:53.676738 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.676687 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls\") pod \"splitter-graph-6396f-548474fcc8-mxqst\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:53.679286 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.679242 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls\") pod \"splitter-graph-6396f-548474fcc8-mxqst\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:53.843708 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.843682 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:53.969884 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:53.969857 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst"] Jul 01 21:18:53.972323 ip-10-0-130-150 kubenswrapper[2558]: W0701 21:18:53.972286 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8785dc6c_a28a_43a4_97f9_904212448f0e.slice/crio-3a626a4718406d807bb2acc5df431e7e500b15e73b17e0d04ad991dc5d454219 WatchSource:0}: Error finding container 3a626a4718406d807bb2acc5df431e7e500b15e73b17e0d04ad991dc5d454219: Status 404 returned error can't find the container with id 3a626a4718406d807bb2acc5df431e7e500b15e73b17e0d04ad991dc5d454219 Jul 01 21:18:54.787547 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:54.787509 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" event={"ID":"8785dc6c-a28a-43a4-97f9-904212448f0e","Type":"ContainerStarted","Data":"d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1"} Jul 01 21:18:54.787720 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:54.787569 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" event={"ID":"8785dc6c-a28a-43a4-97f9-904212448f0e","Type":"ContainerStarted","Data":"3a626a4718406d807bb2acc5df431e7e500b15e73b17e0d04ad991dc5d454219"} Jul 01 21:18:54.787720 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:54.787598 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:18:54.805520 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:18:54.805475 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" podStartSLOduration=2.805460805 podStartE2EDuration="2.805460805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 21:18:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:18:54.803734469 +0000 UTC m=+1919.412177343" watchObservedRunningTime="2026-07-01 21:18:54.805460805 +0000 UTC m=+1919.413903680" Jul 01 21:19:00.797921 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:19:00.797891 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:21:55.935172 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:21:55.935140 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:21:55.940715 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:21:55.940688 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:26:55.958975 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:26:55.958865 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:26:55.965312 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:26:55.965294 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:27:07.661556 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:07.661523 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst"] Jul 01 21:27:07.661941 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:07.661734 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" containerID="cri-o://d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1" gracePeriod=30 Jul 01 21:27:10.795847 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:10.795808 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:27:15.796639 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:15.796555 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:27:20.796548 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:20.796508 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:27:20.797015 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:20.796605 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:27:25.797044 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:25.797000 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:27:30.795853 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:30.795817 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:27:35.796723 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:35.796679 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:27:37.803486 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.803463 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:27:37.934931 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.934857 2558 generic.go:356] "Generic (PLEG): container finished" podID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerID="d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1" exitCode=0 Jul 01 21:27:37.934931 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.934899 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" Jul 01 21:27:37.935106 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.934937 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" event={"ID":"8785dc6c-a28a-43a4-97f9-904212448f0e","Type":"ContainerDied","Data":"d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1"} Jul 01 21:27:37.935106 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.934987 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst" event={"ID":"8785dc6c-a28a-43a4-97f9-904212448f0e","Type":"ContainerDied","Data":"3a626a4718406d807bb2acc5df431e7e500b15e73b17e0d04ad991dc5d454219"} Jul 01 21:27:37.935106 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.935014 2558 scope.go:117] "RemoveContainer" containerID="d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1" Jul 01 21:27:37.939907 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.939887 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls\") pod \"8785dc6c-a28a-43a4-97f9-904212448f0e\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " Jul 01 21:27:37.940034 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.939922 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785dc6c-a28a-43a4-97f9-904212448f0e-openshift-service-ca-bundle\") pod \"8785dc6c-a28a-43a4-97f9-904212448f0e\" (UID: \"8785dc6c-a28a-43a4-97f9-904212448f0e\") " Jul 01 21:27:37.940322 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.940302 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8785dc6c-a28a-43a4-97f9-904212448f0e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "8785dc6c-a28a-43a4-97f9-904212448f0e" (UID: "8785dc6c-a28a-43a4-97f9-904212448f0e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:27:37.942064 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.942041 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8785dc6c-a28a-43a4-97f9-904212448f0e" (UID: "8785dc6c-a28a-43a4-97f9-904212448f0e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:27:37.942875 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.942847 2558 scope.go:117] "RemoveContainer" containerID="d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1" Jul 01 21:27:37.943154 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:27:37.943138 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1\": container with ID starting with d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1 not found: ID does not exist" containerID="d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1" Jul 01 21:27:37.943208 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:37.943163 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1"} err="failed to get container status \"d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1\": rpc error: code = NotFound desc = could not find container \"d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1\": container with ID starting with d49840256364c7664b35e0907ba099401978affb0ab13a7f645098d974b9cdd1 not found: ID does not exist" Jul 01 21:27:38.040742 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:38.040710 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8785dc6c-a28a-43a4-97f9-904212448f0e-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:27:38.040742 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:38.040734 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8785dc6c-a28a-43a4-97f9-904212448f0e-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:27:38.255214 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:38.255185 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst"] Jul 01 21:27:38.258786 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:38.258763 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6396f-548474fcc8-mxqst"] Jul 01 21:27:39.849159 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:27:39.849126 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" path="/var/lib/kubelet/pods/8785dc6c-a28a-43a4-97f9-904212448f0e/volumes" Jul 01 21:31:55.983085 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:31:55.982959 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:31:55.989799 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:31:55.989779 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:34:21.514660 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:21.514623 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt"] Jul 01 21:34:21.515200 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:21.514846 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" containerID="cri-o://b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb" gracePeriod=30 Jul 01 21:34:24.582771 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:24.582733 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:34:29.582991 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:29.582952 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:34:34.582491 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:34.582452 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:34:34.582890 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:34.582556 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:34:38.390026 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:38.389998 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:39.179110 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:39.179086 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:39.583184 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:39.583149 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:34:39.971855 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:39.971767 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:40.749355 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:40.749322 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:41.538117 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:41.538085 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:42.317832 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:42.317797 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:43.112559 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:43.112525 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:43.900323 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:43.900287 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:44.582469 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:44.582430 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:34:44.675292 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:44.675247 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:45.456528 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:45.456504 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:46.243324 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:46.243284 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:47.034979 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:47.034952 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:47.815082 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:47.815048 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:48.645770 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:48.645730 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-ce993-845959cd86-5gstt_f754b68d-c342-4b58-a248-c68fbc970586/switch-graph-ce993/0.log" Jul 01 21:34:49.583239 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:49.583197 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 21:34:51.667779 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.667757 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:34:51.704848 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.704813 2558 generic.go:356] "Generic (PLEG): container finished" podID="f754b68d-c342-4b58-a248-c68fbc970586" containerID="b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb" exitCode=0 Jul 01 21:34:51.705002 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.704850 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" Jul 01 21:34:51.705002 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.704867 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" event={"ID":"f754b68d-c342-4b58-a248-c68fbc970586","Type":"ContainerDied","Data":"b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb"} Jul 01 21:34:51.705002 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.704922 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt" event={"ID":"f754b68d-c342-4b58-a248-c68fbc970586","Type":"ContainerDied","Data":"9580379ef215816ad47bcb0bc884332cdafa7ab0db5366d6b4a1b482f42d1c57"} Jul 01 21:34:51.705002 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.704947 2558 scope.go:117] "RemoveContainer" containerID="b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb" Jul 01 21:34:51.713048 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.713030 2558 scope.go:117] "RemoveContainer" containerID="b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb" Jul 01 21:34:51.713307 ip-10-0-130-150 kubenswrapper[2558]: E0701 21:34:51.713287 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb\": container with ID starting with b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb not found: ID does not exist" containerID="b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb" Jul 01 21:34:51.713377 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.713314 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb"} err="failed to get container status \"b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb\": rpc error: code = NotFound desc = could not find container \"b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb\": container with ID starting with b5f070d8e96364a607b43aea9fab88b4662bc62fa0ee19d035eb01f54d7b50fb not found: ID does not exist" Jul 01 21:34:51.748473 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.748444 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f754b68d-c342-4b58-a248-c68fbc970586-proxy-tls\") pod \"f754b68d-c342-4b58-a248-c68fbc970586\" (UID: \"f754b68d-c342-4b58-a248-c68fbc970586\") " Jul 01 21:34:51.748609 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.748539 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f754b68d-c342-4b58-a248-c68fbc970586-openshift-service-ca-bundle\") pod \"f754b68d-c342-4b58-a248-c68fbc970586\" (UID: \"f754b68d-c342-4b58-a248-c68fbc970586\") " Jul 01 21:34:51.748913 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.748887 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f754b68d-c342-4b58-a248-c68fbc970586-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "f754b68d-c342-4b58-a248-c68fbc970586" (UID: "f754b68d-c342-4b58-a248-c68fbc970586"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 21:34:51.750454 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.750433 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f754b68d-c342-4b58-a248-c68fbc970586-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f754b68d-c342-4b58-a248-c68fbc970586" (UID: "f754b68d-c342-4b58-a248-c68fbc970586"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 21:34:51.849388 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.849364 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f754b68d-c342-4b58-a248-c68fbc970586-openshift-service-ca-bundle\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:34:51.849388 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:51.849388 2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f754b68d-c342-4b58-a248-c68fbc970586-proxy-tls\") on node \"ip-10-0-130-150.ec2.internal\" DevicePath \"\"" Jul 01 21:34:52.019852 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:52.019813 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt"] Jul 01 21:34:52.024118 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:52.024097 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-ce993-845959cd86-5gstt"] Jul 01 21:34:53.843703 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:53.843670 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f754b68d-c342-4b58-a248-c68fbc970586" path="/var/lib/kubelet/pods/f754b68d-c342-4b58-a248-c68fbc970586/volumes" Jul 01 21:34:55.630663 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:55.630634 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2ntmn_1172c283-88c3-407e-8c99-4af696b5984c/global-pull-secret-syncer/0.log" Jul 01 21:34:55.821483 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:55.821456 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zmn82_4d6d0ff1-f7c9-4bda-a541-3eb3abf901f4/konnectivity-agent/0.log" Jul 01 21:34:55.840271 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:34:55.840224 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-150.ec2.internal_c3f741d9ca74dc13685dec19f2b0ff2d/haproxy/0.log" Jul 01 21:35:00.150237 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:00.150207 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85c9c87d46-qzd6s_999102f7-ea92-4cf4-a889-4cf989dbca4a/metrics-server/0.log" Jul 01 21:35:00.195011 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:00.194989 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-t45xb_531e256d-55e5-4517-a876-90925897cb2e/monitoring-plugin/0.log" Jul 01 21:35:00.226660 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:00.226637 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fvf4_38dfda6f-4326-44d6-a494-3d40212210c6/node-exporter/0.log" Jul 01 21:35:00.251231 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:00.251206 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fvf4_38dfda6f-4326-44d6-a494-3d40212210c6/kube-rbac-proxy/0.log" Jul 01 21:35:00.275120 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:00.275100 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fvf4_38dfda6f-4326-44d6-a494-3d40212210c6/init-textfile/0.log" Jul 01 21:35:00.927622 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:00.927594 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5694969698-8zwdm_a64e2477-7f8f-4ed4-b0bd-51f6f2f31252/thanos-query/0.log" Jul 01 21:35:00.952114 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:00.952092 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5694969698-8zwdm_a64e2477-7f8f-4ed4-b0bd-51f6f2f31252/kube-rbac-proxy-web/0.log" Jul 01 21:35:00.979241 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:00.979214 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5694969698-8zwdm_a64e2477-7f8f-4ed4-b0bd-51f6f2f31252/kube-rbac-proxy/0.log" Jul 01 21:35:01.000384 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:01.000363 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5694969698-8zwdm_a64e2477-7f8f-4ed4-b0bd-51f6f2f31252/prom-label-proxy/0.log" Jul 01 21:35:01.021855 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:01.021836 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5694969698-8zwdm_a64e2477-7f8f-4ed4-b0bd-51f6f2f31252/kube-rbac-proxy-rules/0.log" Jul 01 21:35:01.046088 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:01.046070 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5694969698-8zwdm_a64e2477-7f8f-4ed4-b0bd-51f6f2f31252/kube-rbac-proxy-metrics/0.log" Jul 01 21:35:03.340686 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:03.340656 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-549ffd5ff4-n6rsd_5c77a363-eaee-4df8-ada1-2a6b6d4f6dc1/console/0.log" Jul 01 21:35:04.452212 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.452181 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4"] Jul 01 21:35:04.452591 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.452541 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" Jul 01 21:35:04.452591 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.452553 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" Jul 01 21:35:04.452591 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.452562 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" Jul 01 21:35:04.452591 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.452568 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" Jul 01 21:35:04.452718 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.452620 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f754b68d-c342-4b58-a248-c68fbc970586" containerName="switch-graph-ce993" Jul 01 21:35:04.452718 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.452629 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8785dc6c-a28a-43a4-97f9-904212448f0e" containerName="splitter-graph-6396f" Jul 01 21:35:04.455224 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.455203 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.457454 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.457433 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4rws2\"/\"openshift-service-ca.crt\"" Jul 01 21:35:04.457577 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.457541 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4rws2\"/\"kube-root-ca.crt\"" Jul 01 21:35:04.458161 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.458146 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4rws2\"/\"default-dockercfg-dcx4p\"" Jul 01 21:35:04.465469 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.465451 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4"] Jul 01 21:35:04.556839 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.556794 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59frj\" (UniqueName: \"kubernetes.io/projected/34509e66-79ca-4d22-8ca0-d005c574ff0d-kube-api-access-59frj\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.556839 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.556850 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-proc\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.557038 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.556867 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-lib-modules\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.557038 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.556939 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-podres\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.557038 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.556974 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-sys\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.657936 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.657895 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-lib-modules\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.657936 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.657944 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-podres\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.658162 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.657973 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-sys\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.658162 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.658016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59frj\" (UniqueName: \"kubernetes.io/projected/34509e66-79ca-4d22-8ca0-d005c574ff0d-kube-api-access-59frj\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.658162 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.658044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-proc\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.658162 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.658078 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-lib-modules\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.658162 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.658101 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-proc\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.658162 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.658090 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-sys\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.658162 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.658121 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34509e66-79ca-4d22-8ca0-d005c574ff0d-podres\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.666338 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.666315 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59frj\" (UniqueName: \"kubernetes.io/projected/34509e66-79ca-4d22-8ca0-d005c574ff0d-kube-api-access-59frj\") pod \"perf-node-gather-daemonset-rq5s4\" (UID: \"34509e66-79ca-4d22-8ca0-d005c574ff0d\") " pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.765615 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.765550 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:04.778293 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.778233 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-85h8x_0ef76ab8-0b86-4d6c-bfb9-965f0fc84219/dns/0.log" Jul 01 21:35:04.798114 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.798091 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-85h8x_0ef76ab8-0b86-4d6c-bfb9-965f0fc84219/kube-rbac-proxy/0.log" Jul 01 21:35:04.872243 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.872217 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8k6wq_6c605213-6328-49f7-b4e0-3a5283e3d27b/dns-node-resolver/0.log" Jul 01 21:35:04.892822 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:04.892794 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4"] Jul 01 21:35:04.895802 ip-10-0-130-150 kubenswrapper[2558]: W0701 21:35:04.895775 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod34509e66_79ca_4d22_8ca0_d005c574ff0d.slice/crio-0e1cc0948a2a433b2d22008c961431ad5744741da1c42d2f88bd7043a83196a8 WatchSource:0}: Error finding container 0e1cc0948a2a433b2d22008c961431ad5744741da1c42d2f88bd7043a83196a8: Status 404 returned error can't find the container with id 0e1cc0948a2a433b2d22008c961431ad5744741da1c42d2f88bd7043a83196a8 Jul 01 21:35:05.376490 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:05.376439 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7996c789b7-qvjxp_9b6fc79e-6522-45bf-9e7c-387c1d5ea1f7/registry/0.log" Jul 01 21:35:05.424458 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:05.424429 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m5fg2_e5b40ef0-7439-42ae-a63e-30d2c415238b/node-ca/0.log" Jul 01 21:35:05.762290 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:05.762176 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" event={"ID":"34509e66-79ca-4d22-8ca0-d005c574ff0d","Type":"ContainerStarted","Data":"655ae839402a6c667f2e04f5a26e5c5403f25bf2a1c52670c62212ab6bd33b3c"} Jul 01 21:35:05.762290 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:05.762234 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:05.762290 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:05.762281 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" event={"ID":"34509e66-79ca-4d22-8ca0-d005c574ff0d","Type":"ContainerStarted","Data":"0e1cc0948a2a433b2d22008c961431ad5744741da1c42d2f88bd7043a83196a8"} Jul 01 21:35:05.780307 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:05.780238 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" podStartSLOduration=1.780222805 podStartE2EDuration="1.780222805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 21:35:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 21:35:05.778624923 +0000 UTC m=+2890.387067798" watchObservedRunningTime="2026-07-01 21:35:05.780222805 +0000 UTC m=+2890.388665674" Jul 01 21:35:06.571347 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:06.571318 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h4t7s_b0ab5d3e-ed68-4d5e-8bf8-a82ee19bb629/serve-healthcheck-canary/0.log" Jul 01 21:35:07.057123 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:07.057044 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dc864_e5117067-8f06-466b-aa57-be171670744e/kube-rbac-proxy/0.log" Jul 01 21:35:07.076040 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:07.076016 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dc864_e5117067-8f06-466b-aa57-be171670744e/exporter/0.log" Jul 01 21:35:07.094924 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:07.094898 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dc864_e5117067-8f06-466b-aa57-be171670744e/extractor/0.log" Jul 01 21:35:09.173942 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:09.173913 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-67485586b-4jpsp_5e81df96-9892-41da-9516-0d51508b1f7a/manager/0.log" Jul 01 21:35:09.193528 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:09.193506 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-7d672_6d498d32-13ed-4938-ab98-77a4137dbe07/manager/0.log" Jul 01 21:35:09.214845 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:09.214824 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-pn8dm_2ac30cbf-35b3-4280-9df0-7fd99197ce8e/server/0.log" Jul 01 21:35:09.302115 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:09.302088 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-j6ztm_2c34797a-6aef-4abf-8d85-141aab48f601/manager/0.log" Jul 01 21:35:11.775690 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:11.775663 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4rws2/perf-node-gather-daemonset-rq5s4" Jul 01 21:35:14.239660 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.239635 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-64tzn_cf1d7d59-659b-43a0-ad53-cee5333fd5fe/kube-multus/0.log" Jul 01 21:35:14.270923 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.270894 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8wkrd_2212862b-69c4-4886-a071-54cf439b9fd0/kube-multus-additional-cni-plugins/0.log" Jul 01 21:35:14.293581 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.293558 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8wkrd_2212862b-69c4-4886-a071-54cf439b9fd0/egress-router-binary-copy/0.log" Jul 01 21:35:14.317725 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.317701 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8wkrd_2212862b-69c4-4886-a071-54cf439b9fd0/cni-plugins/0.log" Jul 01 21:35:14.338875 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.338803 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8wkrd_2212862b-69c4-4886-a071-54cf439b9fd0/bond-cni-plugin/0.log" Jul 01 21:35:14.365102 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.365082 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8wkrd_2212862b-69c4-4886-a071-54cf439b9fd0/routeoverride-cni/0.log" Jul 01 21:35:14.386741 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.386723 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8wkrd_2212862b-69c4-4886-a071-54cf439b9fd0/whereabouts-cni-bincopy/0.log" Jul 01 21:35:14.408690 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.408668 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8wkrd_2212862b-69c4-4886-a071-54cf439b9fd0/whereabouts-cni/0.log" Jul 01 21:35:14.868667 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.868642 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kkfpv_220e4077-2a30-4706-863a-6ec30801b3f3/network-metrics-daemon/0.log" Jul 01 21:35:14.888661 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:14.888639 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kkfpv_220e4077-2a30-4706-863a-6ec30801b3f3/kube-rbac-proxy/0.log" Jul 01 21:35:15.716690 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.716661 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-controller/0.log" Jul 01 21:35:15.735342 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.735317 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/0.log" Jul 01 21:35:15.735478 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.735466 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovn-acl-logging/1.log" Jul 01 21:35:15.751493 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.751473 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/kube-rbac-proxy-node/0.log" Jul 01 21:35:15.771172 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.771154 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/kube-rbac-proxy-ovn-metrics/0.log" Jul 01 21:35:15.793860 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.793839 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/northd/0.log" Jul 01 21:35:15.813272 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.813238 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/nbdb/0.log" Jul 01 21:35:15.834560 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.834539 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/sbdb/0.log" Jul 01 21:35:15.883996 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:15.883969 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6r267_46646e21-8a7b-4443-98c5-5e5556d69ceb/ovnkube-controller/0.log" Jul 01 21:35:17.379213 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:17.379185 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-xj5kn_af091fbd-26ff-4648-9042-fa91ba493331/network-check-target-container/0.log" Jul 01 21:35:18.268051 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:18.268023 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-9k6kh_c6a61d92-4e50-4935-9291-4a42de8fb80e/iptables-alerter/0.log" Jul 01 21:35:18.873523 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:18.873473 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-4k29z_98ed8b15-b9a4-45e6-9b67-ad39e7abc7aa/tuned/0.log" Jul 01 21:35:22.176147 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:22.176113 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-9rjlz_a06cf41c-de16-46c4-a552-ffb04f2bdbfe/csi-driver/0.log" Jul 01 21:35:22.200048 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:22.200021 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-9rjlz_a06cf41c-de16-46c4-a552-ffb04f2bdbfe/csi-node-driver-registrar/0.log" Jul 01 21:35:22.222502 ip-10-0-130-150 kubenswrapper[2558]: I0701 21:35:22.222478 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-9rjlz_a06cf41c-de16-46c4-a552-ffb04f2bdbfe/csi-liveness-probe/0.log"