Jul 01 18:06:29.601120 ip-10-0-136-152 systemd[1]: Starting Kubernetes Kubelet... Jul 01 18:06:31.174281 ip-10-0-136-152 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:06:31.174281 ip-10-0-136-152 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 18:06:31.174281 ip-10-0-136-152 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:06:31.174281 ip-10-0-136-152 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 18:06:31.174281 ip-10-0-136-152 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:06:31.176260 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.175658 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181498 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181509 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181513 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181516 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181519 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181522 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181525 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181527 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181531 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181533 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181536 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181539 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181541 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181544 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181548 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181551 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181553 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181556 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:06:31.352851 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181559 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:06:31.261880 ip-10-0-136-152 systemd[1]: Started Kubernetes Kubelet. Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181561 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181564 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181573 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181577 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181580 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181582 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181591 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181594 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181597 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181599 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181602 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181605 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181610 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181614 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181617 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181620 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181623 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181625 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:06:31.363200 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181630 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181633 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181636 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181639 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181642 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181645 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181648 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181651 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181654 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181657 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181660 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181662 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181666 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181668 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181671 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181674 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181676 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181679 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181681 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181684 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:06:31.364432 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181687 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181689 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181692 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181694 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181697 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181701 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181704 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181707 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181710 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181713 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181717 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181720 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181722 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181725 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181728 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181730 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181733 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181735 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181738 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181741 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:06:31.365253 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181743 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181746 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181749 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181758 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181760 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181763 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181765 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181784 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181788 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181791 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181794 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181796 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181799 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181802 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181804 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181807 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181809 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181812 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181815 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181818 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:06:31.366275 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181820 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.181823 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182587 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182593 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182596 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182599 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182602 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182605 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182608 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182611 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182614 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182616 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182619 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182622 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182624 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182627 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182635 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182640 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182643 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182647 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:06:31.368201 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182649 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182652 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182654 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182664 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182667 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182670 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182672 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182675 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182678 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182680 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182683 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182685 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182688 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182690 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182695 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182698 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182701 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182704 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182707 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182711 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:06:31.368886 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182714 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182718 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182722 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182725 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182727 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182730 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182733 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182735 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182738 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182747 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182750 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182752 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182755 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182757 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182760 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182762 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182765 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182784 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182788 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:06:31.369497 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182792 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182795 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182799 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182802 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182806 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182810 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182814 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182817 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182820 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182824 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182828 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182832 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182835 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182839 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182843 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182846 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182850 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182854 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182858 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182862 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:06:31.370161 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182866 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182870 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182875 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182888 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182893 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182898 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182904 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182908 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182912 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182916 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182920 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182924 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182928 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182932 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182935 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182938 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182942 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182946 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182949 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:06:31.370802 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182953 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182956 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.182960 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183417 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183424 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183426 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183429 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183431 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183434 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183437 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183439 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183442 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183445 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183448 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183452 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183457 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183460 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183463 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183465 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:06:31.371390 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183469 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183472 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183474 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183477 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183481 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183485 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183488 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183491 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183494 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183497 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183499 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183504 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183507 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183510 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183512 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183515 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183517 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183520 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183523 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183525 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:06:31.372177 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183527 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183530 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183533 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183535 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183538 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183540 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183543 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183545 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183549 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183551 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183554 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183557 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183563 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183566 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183568 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183571 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183573 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183576 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183578 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183581 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:06:31.372800 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183584 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183586 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183589 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183592 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183594 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183597 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183599 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183602 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183604 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183606 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183609 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183612 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183614 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183617 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183619 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183622 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183624 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183627 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183632 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:06:31.373434 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183635 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183639 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183642 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183645 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183648 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183652 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183655 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183657 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183660 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183663 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183665 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183668 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183670 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183673 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183676 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183678 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183681 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183684 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183686 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183689 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183691 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:06:31.373988 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183694 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183696 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.183699 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183787 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183796 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183804 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183808 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183812 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183815 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183819 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183827 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183830 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183835 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183838 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183842 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183845 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183850 2572 flags.go:64] FLAG: --cgroup-root="" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183857 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183860 2572 flags.go:64] FLAG: --client-ca-file="" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183863 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183866 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183871 2572 flags.go:64] FLAG: --cluster-domain="" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183874 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183877 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 18:06:31.576175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183880 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183883 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183887 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183907 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183911 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183915 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183918 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183921 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183924 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183927 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183930 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183934 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183937 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183940 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183943 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183946 2572 flags.go:64] FLAG: --enable-server="true" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183948 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183953 2572 flags.go:64] FLAG: --event-burst="100" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183957 2572 flags.go:64] FLAG: --event-qps="50" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183959 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183964 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183967 2572 flags.go:64] FLAG: --eviction-hard="" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183971 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183974 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183978 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 18:06:31.577267 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183982 2572 flags.go:64] FLAG: --eviction-soft="" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183985 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183988 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183990 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183993 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183996 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.183999 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184002 2572 flags.go:64] FLAG: --feature-gates="" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184005 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184008 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184011 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184014 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184018 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184021 2572 flags.go:64] FLAG: --help="false" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184024 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184027 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184030 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184033 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184036 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184039 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184042 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184045 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184047 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 18:06:31.578001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184050 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184053 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184056 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184058 2572 flags.go:64] FLAG: --kube-reserved="" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184063 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184066 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184069 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184072 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184076 2572 flags.go:64] FLAG: --lock-file="" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184078 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184082 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184085 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184090 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184093 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184096 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184099 2572 flags.go:64] FLAG: --logging-format="text" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184102 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184105 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184108 2572 flags.go:64] FLAG: --manifest-url="" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184111 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184115 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184118 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184122 2572 flags.go:64] FLAG: --max-pods="110" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184125 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184128 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 18:06:31.578705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184131 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184134 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184137 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184140 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184143 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184150 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184153 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184156 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184159 2572 flags.go:64] FLAG: --pod-cidr="" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184162 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184168 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184170 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184175 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184178 2572 flags.go:64] FLAG: --port="10250" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184181 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184185 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a45488b9a2ddf9db" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184188 2572 flags.go:64] FLAG: --qos-reserved="" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184191 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184195 2572 flags.go:64] FLAG: --register-node="true" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184198 2572 flags.go:64] FLAG: --register-with-taints="" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184201 2572 flags.go:64] FLAG: --registry-burst="10" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184204 2572 flags.go:64] FLAG: --registry-qps="5" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184207 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184209 2572 flags.go:64] FLAG: --reserved-memory="" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184213 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 18:06:31.579381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184216 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184219 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184222 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184225 2572 flags.go:64] FLAG: --runonce="false" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184228 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184231 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184234 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184237 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184239 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184242 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184245 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184248 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184251 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184254 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184256 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184259 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184262 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184265 2572 flags.go:64] FLAG: --system-cgroups="" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184268 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184275 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184278 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184280 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184285 2572 flags.go:64] FLAG: --tls-min-version="" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184288 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184291 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 18:06:31.580153 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184294 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184298 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184301 2572 flags.go:64] FLAG: --v="2" Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184305 2572 flags.go:64] FLAG: --version="false" Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184309 2572 flags.go:64] FLAG: --vmodule="" Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184313 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.184316 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184433 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184437 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184440 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184443 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184446 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184449 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184451 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184454 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184457 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184459 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184462 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184464 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184467 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184469 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184472 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:06:31.580858 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184474 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184477 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184479 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184483 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184488 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184491 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184494 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184499 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184501 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184504 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184507 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184511 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184515 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184517 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184520 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184523 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184525 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184528 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:06:31.581498 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184531 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184533 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184536 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184538 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184541 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184543 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184545 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184548 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184550 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184553 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184555 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184558 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184560 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184562 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184565 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184567 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184570 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184574 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184578 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184581 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:06:31.582058 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184584 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184588 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184591 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184594 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184596 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184599 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184602 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184608 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184610 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184613 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184615 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184618 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184620 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184623 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184625 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184628 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184630 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184633 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184635 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184638 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:06:31.582648 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184640 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184643 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184645 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184647 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184650 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184652 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184655 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184657 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184660 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184662 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184666 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184669 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184671 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184675 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184678 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184680 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184683 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184685 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184688 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184691 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:06:31.583197 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184693 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184696 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184698 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184701 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184704 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.184706 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.185604 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.192679 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.192692 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.192711 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.192720 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192796 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192801 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192805 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:06:31.583748 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192807 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192810 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192813 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192816 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192818 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192821 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192823 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192826 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192829 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192834 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192836 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192842 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192845 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192848 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192851 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192854 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192857 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192859 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192863 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192866 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:06:31.584276 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192868 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192871 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192875 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192878 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192881 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192883 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192886 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192888 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192891 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192893 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192896 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192898 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192901 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192903 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192906 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192908 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192911 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192913 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192916 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:06:31.584863 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192919 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192921 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192927 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192930 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192934 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192937 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192939 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192942 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192945 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192947 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192950 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192953 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192962 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192965 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192967 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192970 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192973 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192975 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192978 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192980 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:06:31.585516 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192983 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192985 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192987 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192990 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192992 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192995 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.192998 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193000 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193003 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193005 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193008 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193010 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193013 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193015 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193019 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193022 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193026 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193028 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193031 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193035 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:06:31.586107 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193041 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193044 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193047 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193049 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193052 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193061 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193064 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193066 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193069 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193071 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193074 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193076 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193079 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193081 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193084 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193086 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:06:31.586710 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193089 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.193094 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193226 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193231 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193235 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193239 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193242 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193245 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193247 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193250 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193252 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193256 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193260 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193262 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:06:31.587182 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193265 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193268 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193270 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193273 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193275 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193278 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193280 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193283 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193291 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193293 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193302 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193305 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193307 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193310 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193312 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193315 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193317 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193320 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193322 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193325 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:06:31.587598 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193327 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193330 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193333 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193335 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193338 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193340 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193342 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193345 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193347 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193351 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193355 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193357 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193359 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193362 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193364 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193367 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193370 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193372 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193374 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193377 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:06:31.588378 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193379 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193391 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193394 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193397 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193401 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193403 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193405 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193409 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193412 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193415 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193418 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193420 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193423 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193425 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193428 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193430 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193433 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193435 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193438 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:06:31.588985 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193440 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193442 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193446 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193450 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193452 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193455 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193457 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193460 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193462 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193465 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193467 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193470 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193472 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193475 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193477 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193485 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193488 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193491 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193493 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193496 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:06:31.801528 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193498 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193501 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193503 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193506 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193508 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193511 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193513 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:31.193515 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.193520 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.193642 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.195790 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.197268 2572 server.go:1019] "Starting client certificate rotation" Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.197374 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 18:06:31.802870 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.197413 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 18:06:31.803261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.200264 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 18:06:31.803261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.208762 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 18:06:31.803261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.219922 2572 log.go:25] "Validated CRI v1 runtime API" Jul 01 18:06:31.803261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.226059 2572 log.go:25] "Validated CRI v1 image API" Jul 01 18:06:31.803261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.226808 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 18:06:31.803261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.229443 2572 fs.go:135] Filesystem UUIDs: map[592d12dc-369d-4210-a078-be6af0d90038:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 93addf4d-53e3-400b-b3ee-907818d8b46f:/dev/nvme0n1p3] Jul 01 18:06:31.803261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.229459 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.233700 2572 manager.go:216] Machine: {Timestamp:2026-07-01 18:06:31.233235076 +0000 UTC m=+0.374418439 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099192 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28743f69b4c8ce9726f2d0b3ec8bc9 SystemUUID:ec28743f-69b4-c8ce-9726-f2d0b3ec8bc9 BootID:b3516258-f60c-4e01-8bd4-9c2671176cd1 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2a:d1:8e:e4:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2a:d1:8e:e4:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:81:60:11:17:98 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.233827 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.233928 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.235822 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.236980 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.237012 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-152.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.241227 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.241237 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.241245 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.241256 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.242083 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.246413 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.246539 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.246569 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.246588 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.246599 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 18:06:31.803628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.246608 2572 kubelet.go:398] "Adding apiserver pod source" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.246617 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.247033 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.247067 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.249784 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.251337 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.251364 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.251511 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.251523 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.251529 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.251534 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.251542 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.251548 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.252167 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.252179 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.252187 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.252193 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.252206 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.252216 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.256944 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.257103 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-59nr6" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.257245 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.258712 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.258857 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-152.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.258870 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-152.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.260837 2572 server.go:1268] "Started kubelet" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.260897 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.260911 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.260957 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.262069 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-59nr6" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.262256 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.264091 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.267985 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-152.ec2.internal.18be3ca730c5bcf2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-152.ec2.internal,UID:ip-10-0-136-152.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-152.ec2.internal,},FirstTimestamp:2026-07-01 18:06:31.260765426 +0000 UTC m=+0.401948788,LastTimestamp:2026-07-01 18:06:31.260765426 +0000 UTC m=+0.401948788,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-152.ec2.internal,}" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.270574 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.270581 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.270693 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.271097 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.271145 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.271158 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.271248 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 01 18:06:31.804316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.271322 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.273925 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.273969 2572 factory.go:55] Registering systemd factory Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.273982 2572 factory.go:223] Registration of the systemd container factory successfully Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.274186 2572 factory.go:153] Registering CRI-O factory Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.274198 2572 factory.go:223] Registration of the crio container factory successfully Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.274239 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.274263 2572 factory.go:103] Registering Raw factory Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.274273 2572 manager.go:1196] Started watching for new ooms in manager Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.274636 2572 manager.go:349] Starting recovery of all containers Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.279294 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.282930 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.288439 2572 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-152.ec2.internal\" not found" node="ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.289614 2572 manager.go:354] Recovery completed Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.293912 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.295654 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.295679 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.295689 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.296222 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.296231 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.296231 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.296260 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.297317 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.297349 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.297366 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.297825 2572 policy_none.go:49] "None policy: Start" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.297856 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.297871 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.299113 2572 policy_none.go:47] "Start" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.338341 2572 manager.go:337] "Starting Device Plugin manager" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.338367 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.338377 2572 server.go:87] "Starting device plugin registration server" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.338629 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.338643 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.338730 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.338827 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.338838 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.339299 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.339344 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.354998 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.355033 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.355054 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.355084 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.357375 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.439495 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.440608 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.440637 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.440648 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.440675 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.805814 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.449192 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.449213 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-152.ec2.internal\": node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.455406 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal"] Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.455466 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.456951 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.456975 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.456990 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.458146 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.458258 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.458295 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.458853 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.458879 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.458891 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.459552 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.459579 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.459593 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.460679 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.460701 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.461382 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.461408 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.461422 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.461960 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.471464 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cb0d70dff609cb6a82e1b6ed7c4446d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal\" (UID: \"4cb0d70dff609cb6a82e1b6ed7c4446d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.471488 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a4e6868fe585dc3b6f2f3608909f610d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-152.ec2.internal\" (UID: \"a4e6868fe585dc3b6f2f3608909f610d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.471506 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4cb0d70dff609cb6a82e1b6ed7c4446d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal\" (UID: \"4cb0d70dff609cb6a82e1b6ed7c4446d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.480405 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-152.ec2.internal\" not found" node="ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.484503 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-152.ec2.internal\" not found" node="ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.562408 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.571815 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4cb0d70dff609cb6a82e1b6ed7c4446d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal\" (UID: \"4cb0d70dff609cb6a82e1b6ed7c4446d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.807176 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.571841 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cb0d70dff609cb6a82e1b6ed7c4446d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal\" (UID: \"4cb0d70dff609cb6a82e1b6ed7c4446d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.808051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.571862 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a4e6868fe585dc3b6f2f3608909f610d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-152.ec2.internal\" (UID: \"a4e6868fe585dc3b6f2f3608909f610d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.808051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.571911 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a4e6868fe585dc3b6f2f3608909f610d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-152.ec2.internal\" (UID: \"a4e6868fe585dc3b6f2f3608909f610d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.808051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.571911 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4cb0d70dff609cb6a82e1b6ed7c4446d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal\" (UID: \"4cb0d70dff609cb6a82e1b6ed7c4446d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.808051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.575843 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cb0d70dff609cb6a82e1b6ed7c4446d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal\" (UID: \"4cb0d70dff609cb6a82e1b6ed7c4446d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.808051 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.662837 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:31.808051 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.763466 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:31.808051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.783848 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:31.808051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:31.787670 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" Jul 01 18:06:32.006811 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.864559 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.006811 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:31.965010 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.185886 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:32.065489 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.185886 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:32.166106 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.361977 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.196564 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 18:06:32.361977 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.196696 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 18:06:32.361977 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.196739 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 18:06:32.361977 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:32.266814 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.361977 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.266844 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 18:01:31 +0000 UTC" deadline="2028-03-05 23:34:44.095385693 +0000 UTC" Jul 01 18:06:32.361977 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.266971 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14717h28m11.828420331s" Jul 01 18:06:32.550255 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:32.367820 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.550255 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:32.412306 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb0d70dff609cb6a82e1b6ed7c4446d.slice/crio-f4d5bdd8285619441164df36cac89ff3c37c3e78716ed946e74730628ce0e0e9 WatchSource:0}: Error finding container f4d5bdd8285619441164df36cac89ff3c37c3e78716ed946e74730628ce0e0e9: Status 404 returned error can't find the container with id f4d5bdd8285619441164df36cac89ff3c37c3e78716ed946e74730628ce0e0e9 Jul 01 18:06:32.550255 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:32.412995 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e6868fe585dc3b6f2f3608909f610d.slice/crio-00614df8820e98076bd443364259e7162235c66352c80a036e77cfd6c9da1f8e WatchSource:0}: Error finding container 00614df8820e98076bd443364259e7162235c66352c80a036e77cfd6c9da1f8e: Status 404 returned error can't find the container with id 00614df8820e98076bd443364259e7162235c66352c80a036e77cfd6c9da1f8e Jul 01 18:06:32.550255 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.417225 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 18:06:32.550255 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:32.468376 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.747211 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:32.568848 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.747211 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:32.669396 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-152.ec2.internal\" not found" Jul 01 18:06:32.747211 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.684930 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:06:32.747211 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.717797 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:06:32.901647 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.770786 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" Jul 01 18:06:32.901647 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.782495 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 18:06:32.901647 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.783625 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" Jul 01 18:06:32.901647 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:32.790443 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 18:06:33.247096 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.247066 2572 apiserver.go:52] "Watching apiserver" Jul 01 18:06:33.254107 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.254077 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 18:06:33.254660 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.254636 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-vhlpx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl","openshift-image-registry/node-ca-lnrs5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal","openshift-multus/multus-additional-cni-plugins-sftwb","openshift-multus/multus-m2r5j","openshift-network-operator/iptables-alerter-mrltk","openshift-ovn-kubernetes/ovnkube-node-xmjfl","kube-system/global-pull-secret-syncer-5jn49","kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal","openshift-cluster-node-tuning-operator/tuned-pl4gm","openshift-dns/node-resolver-f5fv7","openshift-multus/network-metrics-daemon-dr9zg","openshift-network-diagnostics/network-check-target-sddsx"] Jul 01 18:06:33.256433 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.256214 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:33.256433 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.256304 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:33.258274 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.257932 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.259649 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.259076 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.261336 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.260291 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 18:06:33.261336 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.260564 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 18:06:33.261336 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.260580 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 18:06:33.261336 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.260791 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vr9kh\"" Jul 01 18:06:33.261336 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.260824 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:33.261336 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.261070 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 18:06:33.261682 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.261365 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bqn9z\"" Jul 01 18:06:33.262855 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.261943 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.263605 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.263141 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.263605 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.263396 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 18:06:33.264604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.264568 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.265727 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.265891 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.265971 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266169 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266186 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266199 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hkm2d\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266396 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4tfcw\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266469 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266529 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pnvfz\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266585 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 18:06:33.266718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266664 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 18:06:33.267282 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266707 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 18:06:33.267282 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.266910 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.268021 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.268001 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 18:06:33.268221 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.268203 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tts7v\"" Jul 01 18:06:33.268731 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.268375 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 18:06:33.268731 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.268535 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:06:33.268965 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.268948 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 18:06:33.269149 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.269134 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 18:06:33.271031 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.271013 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ntvm8\"" Jul 01 18:06:33.271869 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.271846 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 18:06:33.271869 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.271862 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 18:06:33.273157 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.273138 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 18:06:33.273363 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.273350 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 18:06:33.273854 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.273839 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.273939 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.273918 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:33.276181 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.274277 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 18:06:33.277243 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.277080 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:33.277243 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.277153 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:33.279592 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.279308 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.280763 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280741 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-kubelet\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.280870 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280840 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:33.280930 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280869 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.280930 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280892 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-socket-dir-parent\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.280930 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280915 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-cni-bin\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.281076 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280937 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-cni-bin\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.281076 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280974 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-os-release\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.281076 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280994 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.281076 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.280998 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.281261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281080 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46476d5f-12a3-4783-b432-854f32042355-dbus\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.281261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281110 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-cnibin\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.281261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281140 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-daemon-config\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.281261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281169 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.281261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281218 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.281493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281263 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-cni-multus\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.281493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281307 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-kubelet\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.281493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281367 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-slash\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.281493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281418 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbzzf\" (UniqueName: \"kubernetes.io/projected/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-kube-api-access-rbzzf\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:33.281493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281436 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:06:33.281493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281479 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-ovn\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281516 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281545 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xq58\" (UniqueName: \"kubernetes.io/projected/9be98694-626a-44c6-af62-6c86c68a8d7a-kube-api-access-5xq58\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281580 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-cni-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281598 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281603 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-hostroot\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281637 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-run-netns\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281651 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-systemd\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281667 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovnkube-script-lib\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281682 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49lvr\" (UniqueName: \"kubernetes.io/projected/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-kube-api-access-49lvr\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281684 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s5d6x\"" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281698 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281736 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-cnibin\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.281787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281758 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-cni-binary-copy\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281796 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281867 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-device-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281898 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-os-release\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281925 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-etc-kubernetes\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281956 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/250e8f25-8966-4f14-83ab-1e5b42aeb73a-iptables-alerter-script\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.281990 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-system-cni-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282016 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9418d21-5deb-43b9-aa86-476f01d902e5-agent-certs\") pod \"konnectivity-agent-vhlpx\" (UID: \"a9418d21-5deb-43b9-aa86-476f01d902e5\") " pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9418d21-5deb-43b9-aa86-476f01d902e5-konnectivity-ca\") pod \"konnectivity-agent-vhlpx\" (UID: \"a9418d21-5deb-43b9-aa86-476f01d902e5\") " pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282060 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-host\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282081 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-node-log\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282104 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-cni-netd\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282129 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-system-cni-dir\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.282370 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282154 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282178 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-serviceca\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282674 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-k8s-cni-cncf-io\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282704 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282742 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovnkube-config\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282761 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46476d5f-12a3-4783-b432-854f32042355-kubelet-config\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282802 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282825 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/250e8f25-8966-4f14-83ab-1e5b42aeb73a-host-slash\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282876 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-conf-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282896 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-netns\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282915 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-systemd-units\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282938 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-etc-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.282975 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282978 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-49hlv\"" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283022 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.282998 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gcbx\" (UniqueName: \"kubernetes.io/projected/5f9ea0bd-275c-4941-abea-106b2c01ab01-kube-api-access-9gcbx\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283065 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-cni-binary-copy\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283091 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw2wk\" (UniqueName: \"kubernetes.io/projected/627519ad-b343-43ab-8f0f-9fc101b7cb91-kube-api-access-rw2wk\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283120 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5snb\" (UniqueName: \"kubernetes.io/projected/250e8f25-8966-4f14-83ab-1e5b42aeb73a-kube-api-access-c5snb\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283153 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283185 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5n2k\" (UniqueName: \"kubernetes.io/projected/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-kube-api-access-r5n2k\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283208 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-log-socket\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283240 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283268 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-multus-certs\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283290 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-var-lib-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283312 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-env-overrides\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.283500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283334 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovn-node-metrics-cert\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.284161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.283599 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 18:06:33.285651 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.285620 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 18:06:33.308578 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.308536 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xnt54" Jul 01 18:06:33.316308 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.316288 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xnt54" Jul 01 18:06:33.366545 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.366441 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" event={"ID":"4cb0d70dff609cb6a82e1b6ed7c4446d","Type":"ContainerStarted","Data":"f4d5bdd8285619441164df36cac89ff3c37c3e78716ed946e74730628ce0e0e9"} Jul 01 18:06:33.370610 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.370490 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" event={"ID":"a4e6868fe585dc3b6f2f3608909f610d","Type":"ContainerStarted","Data":"00614df8820e98076bd443364259e7162235c66352c80a036e77cfd6c9da1f8e"} Jul 01 18:06:33.372129 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.372107 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 18:06:33.383522 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383487 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8528571c-8df9-4351-b771-3d37effe01ca-hosts-file\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.383653 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383527 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-system-cni-dir\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.383653 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.383653 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383601 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-serviceca\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.383653 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383628 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-k8s-cni-cncf-io\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.383653 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383637 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-system-cni-dir\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383656 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383682 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovnkube-config\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383705 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46476d5f-12a3-4783-b432-854f32042355-kubelet-config\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383764 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383808 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-kubernetes\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383834 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/250e8f25-8966-4f14-83ab-1e5b42aeb73a-host-slash\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-conf-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383883 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-run\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383912 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-tmp\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383934 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8528571c-8df9-4351-b771-3d37effe01ca-tmp-dir\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383958 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-netns\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.383978 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.383983 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-systemd-units\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384005 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-etc-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-var-lib-kubelet\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384059 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gcbx\" (UniqueName: \"kubernetes.io/projected/5f9ea0bd-275c-4941-abea-106b2c01ab01-kube-api-access-9gcbx\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384083 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-cni-binary-copy\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384106 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-serviceca\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384107 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw2wk\" (UniqueName: \"kubernetes.io/projected/627519ad-b343-43ab-8f0f-9fc101b7cb91-kube-api-access-rw2wk\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384165 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5snb\" (UniqueName: \"kubernetes.io/projected/250e8f25-8966-4f14-83ab-1e5b42aeb73a-kube-api-access-c5snb\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384182 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5n2k\" (UniqueName: \"kubernetes.io/projected/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-kube-api-access-r5n2k\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384214 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-log-socket\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384234 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384242 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384253 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-multus-certs\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384268 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-var-lib-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384283 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-env-overrides\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384298 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovn-node-metrics-cert\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.384604 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384314 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-lib-modules\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384333 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-host\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384355 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-kubelet\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384391 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384393 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-k8s-cni-cncf-io\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384406 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-socket-dir-parent\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384421 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-cni-bin\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384497 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384521 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-netns\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384564 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-systemd-units\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384603 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-etc-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384787 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384916 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-env-overrides\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.384971 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-cni-bin\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385045 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-sys\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385071 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-os-release\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.385456 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385097 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385124 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46476d5f-12a3-4783-b432-854f32042355-dbus\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385149 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-cnibin\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385164 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385175 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-daemon-config\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385202 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385230 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r65pr\" (UniqueName: \"kubernetes.io/projected/8528571c-8df9-4351-b771-3d37effe01ca-kube-api-access-r65pr\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385306 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-cni-binary-copy\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385330 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385357 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-cni-multus\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385411 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-kubelet\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385437 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-slash\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385444 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-cni-bin\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385468 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-systemd\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385486 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46476d5f-12a3-4783-b432-854f32042355-kubelet-config\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385499 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbzzf\" (UniqueName: \"kubernetes.io/projected/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-kube-api-access-rbzzf\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:33.386430 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385527 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-ovn\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385552 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385580 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xq58\" (UniqueName: \"kubernetes.io/projected/9be98694-626a-44c6-af62-6c86c68a8d7a-kube-api-access-5xq58\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385590 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-cnibin\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385594 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovnkube-config\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385605 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-cni-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385405 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385781 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-run-multus-certs\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385795 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-kubelet\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385805 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-cni-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385814 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-log-socket\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385832 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-var-lib-openvswitch\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385830 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-hostroot\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385855 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-slash\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385875 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385898 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-kubelet\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385919 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-hostroot\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.385928 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:33.387368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385934 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-cni-bin\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385985 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.385997 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:06:33.885973875 +0000 UTC m=+3.027157245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386028 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.386045 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386086 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-run-netns\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.386101 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret podName:46476d5f-12a3-4783-b432-854f32042355 nodeName:}" failed. No retries permitted until 2026-07-01 18:06:33.886083456 +0000 UTC m=+3.027266823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret") pod "global-pull-secret-syncer-5jn49" (UID: "46476d5f-12a3-4783-b432-854f32042355") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-run-netns\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386098 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-host-var-lib-cni-multus\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.385934 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-os-release\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386157 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46476d5f-12a3-4783-b432-854f32042355-dbus\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386155 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386240 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/250e8f25-8966-4f14-83ab-1e5b42aeb73a-host-slash\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386254 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-systemd\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386281 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-socket-dir-parent\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386322 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovnkube-script-lib\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysctl-d\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386363 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-systemd\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386392 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49lvr\" (UniqueName: \"kubernetes.io/projected/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-kube-api-access-49lvr\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386393 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-conf-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386413 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-multus-daemon-config\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386437 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-run-ovn\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386465 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386569 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-modprobe-d\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386589 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-run-ovn-kubernetes\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386645 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysctl-conf\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386699 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gmjz\" (UniqueName: \"kubernetes.io/projected/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-kube-api-access-6gmjz\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386739 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-cnibin\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386788 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-cni-binary-copy\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386792 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9ea0bd-275c-4941-abea-106b2c01ab01-cnibin\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386843 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovnkube-script-lib\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386839 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386959 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-device-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.388992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.386995 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-os-release\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387017 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-etc-kubernetes\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387053 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/250e8f25-8966-4f14-83ab-1e5b42aeb73a-iptables-alerter-script\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-system-cni-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387107 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysconfig\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387154 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-tuned\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387182 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9418d21-5deb-43b9-aa86-476f01d902e5-agent-certs\") pod \"konnectivity-agent-vhlpx\" (UID: \"a9418d21-5deb-43b9-aa86-476f01d902e5\") " pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387207 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9418d21-5deb-43b9-aa86-476f01d902e5-konnectivity-ca\") pod \"konnectivity-agent-vhlpx\" (UID: \"a9418d21-5deb-43b9-aa86-476f01d902e5\") " pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387230 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-host\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387258 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-node-log\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387284 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-cni-netd\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387296 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-cni-binary-copy\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387381 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-host\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387385 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f9ea0bd-275c-4941-abea-106b2c01ab01-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387382 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9be98694-626a-44c6-af62-6c86c68a8d7a-device-dir\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387413 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-etc-kubernetes\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387439 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-node-log\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387453 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/627519ad-b343-43ab-8f0f-9fc101b7cb91-host-cni-netd\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.390002 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387466 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-os-release\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.390912 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387520 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-system-cni-dir\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.390912 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.387967 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/250e8f25-8966-4f14-83ab-1e5b42aeb73a-iptables-alerter-script\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.390912 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.389006 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9418d21-5deb-43b9-aa86-476f01d902e5-konnectivity-ca\") pod \"konnectivity-agent-vhlpx\" (UID: \"a9418d21-5deb-43b9-aa86-476f01d902e5\") " pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:33.390912 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.389210 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/627519ad-b343-43ab-8f0f-9fc101b7cb91-ovn-node-metrics-cert\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.390912 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.390632 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9418d21-5deb-43b9-aa86-476f01d902e5-agent-certs\") pod \"konnectivity-agent-vhlpx\" (UID: \"a9418d21-5deb-43b9-aa86-476f01d902e5\") " pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:33.393521 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.393464 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gcbx\" (UniqueName: \"kubernetes.io/projected/5f9ea0bd-275c-4941-abea-106b2c01ab01-kube-api-access-9gcbx\") pod \"multus-additional-cni-plugins-sftwb\" (UID: \"5f9ea0bd-275c-4941-abea-106b2c01ab01\") " pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.395269 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.395161 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw2wk\" (UniqueName: \"kubernetes.io/projected/627519ad-b343-43ab-8f0f-9fc101b7cb91-kube-api-access-rw2wk\") pod \"ovnkube-node-xmjfl\" (UID: \"627519ad-b343-43ab-8f0f-9fc101b7cb91\") " pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.395375 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.395299 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbzzf\" (UniqueName: \"kubernetes.io/projected/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-kube-api-access-rbzzf\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:33.395707 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.395668 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5n2k\" (UniqueName: \"kubernetes.io/projected/9de04a82-ce0c-4dfa-b8e1-1db529bdfbab-kube-api-access-r5n2k\") pod \"node-ca-lnrs5\" (UID: \"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab\") " pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.396094 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.396053 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xq58\" (UniqueName: \"kubernetes.io/projected/9be98694-626a-44c6-af62-6c86c68a8d7a-kube-api-access-5xq58\") pod \"aws-ebs-csi-driver-node-fccpl\" (UID: \"9be98694-626a-44c6-af62-6c86c68a8d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.396729 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.396689 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49lvr\" (UniqueName: \"kubernetes.io/projected/c1b2d4a9-d472-4ac8-a33f-af4d676109ab-kube-api-access-49lvr\") pod \"multus-m2r5j\" (UID: \"c1b2d4a9-d472-4ac8-a33f-af4d676109ab\") " pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.397458 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.397436 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5snb\" (UniqueName: \"kubernetes.io/projected/250e8f25-8966-4f14-83ab-1e5b42aeb73a-kube-api-access-c5snb\") pod \"iptables-alerter-mrltk\" (UID: \"250e8f25-8966-4f14-83ab-1e5b42aeb73a\") " pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.487654 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487614 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-lib-modules\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.487826 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487654 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-host\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.487826 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487738 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-host\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.487826 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487748 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-sys\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.487826 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487803 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:33.487826 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487821 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-sys\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487831 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r65pr\" (UniqueName: \"kubernetes.io/projected/8528571c-8df9-4351-b771-3d37effe01ca-kube-api-access-r65pr\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-systemd\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487908 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysctl-d\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487842 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-lib-modules\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487944 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-modprobe-d\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.487976 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysctl-conf\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488000 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gmjz\" (UniqueName: \"kubernetes.io/projected/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-kube-api-access-6gmjz\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488032 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysconfig\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488041 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-modprobe-d\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488055 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-tuned\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8528571c-8df9-4351-b771-3d37effe01ca-hosts-file\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488120 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-kubernetes\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488148 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-run\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488171 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-tmp\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488210 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8528571c-8df9-4351-b771-3d37effe01ca-tmp-dir\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488235 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-var-lib-kubelet\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488297 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysctl-conf\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488336 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-kubernetes\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488339 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-systemd\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-run\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488335 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-var-lib-kubelet\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488472 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysctl-d\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.488616 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488517 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8528571c-8df9-4351-b771-3d37effe01ca-hosts-file\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.489185 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488792 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8528571c-8df9-4351-b771-3d37effe01ca-tmp-dir\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.489185 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.488859 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-sysconfig\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.491162 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.491130 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-etc-tuned\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.491464 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.491449 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-tmp\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.496911 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.496888 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:06:33.497008 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.496914 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:06:33.497008 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.496928 2572 projected.go:196] Error preparing data for projected volume kube-api-access-szjsq for pod openshift-network-diagnostics/network-check-target-sddsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:33.497008 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.496988 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq podName:9395f543-f981-48e3-8a19-d285b1bc933f nodeName:}" failed. No retries permitted until 2026-07-01 18:06:33.996970242 +0000 UTC m=+3.138153609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-szjsq" (UniqueName: "kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq") pod "network-check-target-sddsx" (UID: "9395f543-f981-48e3-8a19-d285b1bc933f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:33.499543 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.499493 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r65pr\" (UniqueName: \"kubernetes.io/projected/8528571c-8df9-4351-b771-3d37effe01ca-kube-api-access-r65pr\") pod \"node-resolver-f5fv7\" (UID: \"8528571c-8df9-4351-b771-3d37effe01ca\") " pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.500230 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.500208 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gmjz\" (UniqueName: \"kubernetes.io/projected/4d53392f-dc5c-4868-8447-4a5f8c5f6f16-kube-api-access-6gmjz\") pod \"tuned-pl4gm\" (UID: \"4d53392f-dc5c-4868-8447-4a5f8c5f6f16\") " pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.527069 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.527042 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:06:33.582283 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.582251 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" Jul 01 18:06:33.590014 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.589985 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lnrs5" Jul 01 18:06:33.600748 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.600729 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:33.607449 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.607432 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sftwb" Jul 01 18:06:33.615165 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.615145 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m2r5j" Jul 01 18:06:33.624852 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.624832 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mrltk" Jul 01 18:06:33.634632 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.634611 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:33.642586 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.642564 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" Jul 01 18:06:33.649248 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.649228 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f5fv7" Jul 01 18:06:33.654478 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.654461 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:06:33.891668 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.891576 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:33.891668 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:33.891636 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:33.891942 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.891743 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:33.891942 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.891791 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:33.891942 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.891825 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:06:34.891805773 +0000 UTC m=+4.032989126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:33.891942 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:33.891843 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret podName:46476d5f-12a3-4783-b432-854f32042355 nodeName:}" failed. No retries permitted until 2026-07-01 18:06:34.891834263 +0000 UTC m=+4.033017614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret") pod "global-pull-secret-syncer-5jn49" (UID: "46476d5f-12a3-4783-b432-854f32042355") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:34.093161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.093128 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:34.093368 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.093347 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:06:34.093411 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.093376 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:06:34.093411 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.093390 2572 projected.go:196] Error preparing data for projected volume kube-api-access-szjsq for pod openshift-network-diagnostics/network-check-target-sddsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:34.093484 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.093466 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq podName:9395f543-f981-48e3-8a19-d285b1bc933f nodeName:}" failed. No retries permitted until 2026-07-01 18:06:35.093441741 +0000 UTC m=+4.234625098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-szjsq" (UniqueName: "kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq") pod "network-check-target-sddsx" (UID: "9395f543-f981-48e3-8a19-d285b1bc933f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:34.318423 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.318333 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 18:01:33 +0000 UTC" deadline="2028-02-21 19:24:50.199513112 +0000 UTC" Jul 01 18:06:34.318423 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.318378 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14401h18m15.88113685s" Jul 01 18:06:34.339804 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.339719 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod627519ad_b343_43ab_8f0f_9fc101b7cb91.slice/crio-159b24ea2831ebfae44b71b0a8e96a606cbd6e20b01cb0b1e34a83de50bed34e WatchSource:0}: Error finding container 159b24ea2831ebfae44b71b0a8e96a606cbd6e20b01cb0b1e34a83de50bed34e: Status 404 returned error can't find the container with id 159b24ea2831ebfae44b71b0a8e96a606cbd6e20b01cb0b1e34a83de50bed34e Jul 01 18:06:34.354947 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.354922 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9ea0bd_275c_4941_abea_106b2c01ab01.slice/crio-f90e955f64ed0c3a65e2c1c7d19fec3fb750dcd916891f5b1b46dcb497623e91 WatchSource:0}: Error finding container f90e955f64ed0c3a65e2c1c7d19fec3fb750dcd916891f5b1b46dcb497623e91: Status 404 returned error can't find the container with id f90e955f64ed0c3a65e2c1c7d19fec3fb750dcd916891f5b1b46dcb497623e91 Jul 01 18:06:34.355492 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.355464 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:34.355623 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.355599 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:34.355721 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.355652 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d53392f_dc5c_4868_8447_4a5f8c5f6f16.slice/crio-f23a8147801556f761599a81b712395e18bf5b9a55e98395a2f8f69fbda7080e WatchSource:0}: Error finding container f23a8147801556f761599a81b712395e18bf5b9a55e98395a2f8f69fbda7080e: Status 404 returned error can't find the container with id f23a8147801556f761599a81b712395e18bf5b9a55e98395a2f8f69fbda7080e Jul 01 18:06:34.357080 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.357060 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9be98694_626a_44c6_af62_6c86c68a8d7a.slice/crio-1976cbabb735d893b3c9856fa186974b35e8899664299321282b8583e3a967ed WatchSource:0}: Error finding container 1976cbabb735d893b3c9856fa186974b35e8899664299321282b8583e3a967ed: Status 404 returned error can't find the container with id 1976cbabb735d893b3c9856fa186974b35e8899664299321282b8583e3a967ed Jul 01 18:06:34.358572 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.358487 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b2d4a9_d472_4ac8_a33f_af4d676109ab.slice/crio-2c7b63879cae0ec16fa43aec61bea81e2029e028545e72060907459f97d9c375 WatchSource:0}: Error finding container 2c7b63879cae0ec16fa43aec61bea81e2029e028545e72060907459f97d9c375: Status 404 returned error can't find the container with id 2c7b63879cae0ec16fa43aec61bea81e2029e028545e72060907459f97d9c375 Jul 01 18:06:34.360066 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.360049 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de04a82_ce0c_4dfa_b8e1_1db529bdfbab.slice/crio-61dcfbd8b230fa3e2c0c2bbe77ef7041ad3741a72ec8feb18d528089f0ee4db2 WatchSource:0}: Error finding container 61dcfbd8b230fa3e2c0c2bbe77ef7041ad3741a72ec8feb18d528089f0ee4db2: Status 404 returned error can't find the container with id 61dcfbd8b230fa3e2c0c2bbe77ef7041ad3741a72ec8feb18d528089f0ee4db2 Jul 01 18:06:34.361160 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.361135 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod250e8f25_8966_4f14_83ab_1e5b42aeb73a.slice/crio-88a6cfa5b33c1c87bead33b8ac67de6555039e506b1c418a58dc5dc46e843989 WatchSource:0}: Error finding container 88a6cfa5b33c1c87bead33b8ac67de6555039e506b1c418a58dc5dc46e843989: Status 404 returned error can't find the container with id 88a6cfa5b33c1c87bead33b8ac67de6555039e506b1c418a58dc5dc46e843989 Jul 01 18:06:34.361890 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.361854 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9418d21_5deb_43b9_aa86_476f01d902e5.slice/crio-2dece3f88f3c115108fb275fa62f401dc4899e85bfa6e8c71c9042d7dd1182be WatchSource:0}: Error finding container 2dece3f88f3c115108fb275fa62f401dc4899e85bfa6e8c71c9042d7dd1182be: Status 404 returned error can't find the container with id 2dece3f88f3c115108fb275fa62f401dc4899e85bfa6e8c71c9042d7dd1182be Jul 01 18:06:34.362663 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:06:34.362644 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8528571c_8df9_4351_b771_3d37effe01ca.slice/crio-dc4f0e5f8121bfb27ec5e16c0912d104382998f93f284e25ad384360cf16e5ac WatchSource:0}: Error finding container dc4f0e5f8121bfb27ec5e16c0912d104382998f93f284e25ad384360cf16e5ac: Status 404 returned error can't find the container with id dc4f0e5f8121bfb27ec5e16c0912d104382998f93f284e25ad384360cf16e5ac Jul 01 18:06:34.373025 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.373002 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mrltk" event={"ID":"250e8f25-8966-4f14-83ab-1e5b42aeb73a","Type":"ContainerStarted","Data":"88a6cfa5b33c1c87bead33b8ac67de6555039e506b1c418a58dc5dc46e843989"} Jul 01 18:06:34.374337 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.374296 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m2r5j" event={"ID":"c1b2d4a9-d472-4ac8-a33f-af4d676109ab","Type":"ContainerStarted","Data":"2c7b63879cae0ec16fa43aec61bea81e2029e028545e72060907459f97d9c375"} Jul 01 18:06:34.376068 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.376047 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" event={"ID":"9be98694-626a-44c6-af62-6c86c68a8d7a","Type":"ContainerStarted","Data":"1976cbabb735d893b3c9856fa186974b35e8899664299321282b8583e3a967ed"} Jul 01 18:06:34.377132 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.377113 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lnrs5" event={"ID":"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab","Type":"ContainerStarted","Data":"61dcfbd8b230fa3e2c0c2bbe77ef7041ad3741a72ec8feb18d528089f0ee4db2"} Jul 01 18:06:34.378265 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.378243 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" event={"ID":"4d53392f-dc5c-4868-8447-4a5f8c5f6f16","Type":"ContainerStarted","Data":"f23a8147801556f761599a81b712395e18bf5b9a55e98395a2f8f69fbda7080e"} Jul 01 18:06:34.379489 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.379465 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerStarted","Data":"f90e955f64ed0c3a65e2c1c7d19fec3fb750dcd916891f5b1b46dcb497623e91"} Jul 01 18:06:34.381123 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.381104 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"159b24ea2831ebfae44b71b0a8e96a606cbd6e20b01cb0b1e34a83de50bed34e"} Jul 01 18:06:34.382279 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.382233 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f5fv7" event={"ID":"8528571c-8df9-4351-b771-3d37effe01ca","Type":"ContainerStarted","Data":"dc4f0e5f8121bfb27ec5e16c0912d104382998f93f284e25ad384360cf16e5ac"} Jul 01 18:06:34.383336 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.383312 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhlpx" event={"ID":"a9418d21-5deb-43b9-aa86-476f01d902e5","Type":"ContainerStarted","Data":"2dece3f88f3c115108fb275fa62f401dc4899e85bfa6e8c71c9042d7dd1182be"} Jul 01 18:06:34.575686 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.575649 2572 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c31f1f5cf0b2f04acb86acba1c5b6b839a148ba2a82a0ffd2a747464e10d379: pinging container registry quay.io: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c31f1f5cf0b2f04acb86acba1c5b6b839a148ba2a82a0ffd2a747464e10d379" Jul 01 18:06:34.575832 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.575764 2572 kuberuntime_manager.go:1459] "Unhandled Error" err="container node-ca start failed in pod node-ca-lnrs5_openshift-image-registry(9de04a82-ce0c-4dfa-b8e1-1db529bdfbab): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c31f1f5cf0b2f04acb86acba1c5b6b839a148ba2a82a0ffd2a747464e10d379: pinging container registry quay.io: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 01 18:06:34.575832 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.575818 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node-ca\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c31f1f5cf0b2f04acb86acba1c5b6b839a148ba2a82a0ffd2a747464e10d379: pinging container registry quay.io: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="openshift-image-registry/node-ca-lnrs5" podUID="9de04a82-ce0c-4dfa-b8e1-1db529bdfbab" Jul 01 18:06:34.900938 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.900097 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:34.900938 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:34.900178 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:34.900938 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.900338 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:34.900938 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.900399 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret podName:46476d5f-12a3-4783-b432-854f32042355 nodeName:}" failed. No retries permitted until 2026-07-01 18:06:36.900381036 +0000 UTC m=+6.041564390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret") pod "global-pull-secret-syncer-5jn49" (UID: "46476d5f-12a3-4783-b432-854f32042355") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:34.900938 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.900811 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:34.900938 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:34.900858 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:06:36.900842849 +0000 UTC m=+6.042026207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:35.103092 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.102179 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:35.103092 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:35.102603 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:06:35.103092 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:35.102625 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:06:35.103092 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:35.102640 2572 projected.go:196] Error preparing data for projected volume kube-api-access-szjsq for pod openshift-network-diagnostics/network-check-target-sddsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:35.103092 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:35.102700 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq podName:9395f543-f981-48e3-8a19-d285b1bc933f nodeName:}" failed. No retries permitted until 2026-07-01 18:06:37.102679727 +0000 UTC m=+6.243863094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-szjsq" (UniqueName: "kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq") pod "network-check-target-sddsx" (UID: "9395f543-f981-48e3-8a19-d285b1bc933f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:35.318900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.318609 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 18:01:33 +0000 UTC" deadline="2028-02-11 22:26:37.02327345 +0000 UTC" Jul 01 18:06:35.318900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.318634 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14164h20m1.704642604s" Jul 01 18:06:35.358302 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.357852 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:35.358302 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:35.357973 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:35.358302 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.358116 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:35.358302 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:35.358221 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:35.402250 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.402145 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" event={"ID":"a4e6868fe585dc3b6f2f3608909f610d","Type":"ContainerStarted","Data":"d9e43b2085bb92a8b689bb9ecd568b1154bcf8c3f8eadb96cdf93b6df5ffb43c"} Jul 01 18:06:35.405477 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.405447 2572 generic.go:356] "Generic (PLEG): container finished" podID="4cb0d70dff609cb6a82e1b6ed7c4446d" containerID="ac6046d742f3f7b1eb69ef33ce95d22f6c12542c768d2607b53b9676c5e6ab14" exitCode=0 Jul 01 18:06:35.405618 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.405521 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" event={"ID":"4cb0d70dff609cb6a82e1b6ed7c4446d","Type":"ContainerDied","Data":"ac6046d742f3f7b1eb69ef33ce95d22f6c12542c768d2607b53b9676c5e6ab14"} Jul 01 18:06:35.410603 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:35.410569 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node-ca\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c31f1f5cf0b2f04acb86acba1c5b6b839a148ba2a82a0ffd2a747464e10d379\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4c31f1f5cf0b2f04acb86acba1c5b6b839a148ba2a82a0ffd2a747464e10d379: pinging container registry quay.io: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="openshift-image-registry/node-ca-lnrs5" podUID="9de04a82-ce0c-4dfa-b8e1-1db529bdfbab" Jul 01 18:06:35.430695 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:35.430473 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-152.ec2.internal" podStartSLOduration=3.430453806 podStartE2EDuration="3.430453806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:06:35.417221353 +0000 UTC m=+4.558404725" watchObservedRunningTime="2026-07-01 18:06:35.430453806 +0000 UTC m=+4.571637176" Jul 01 18:06:36.356335 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:36.355661 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:36.356335 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:36.355818 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:36.419856 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:36.419822 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" event={"ID":"4cb0d70dff609cb6a82e1b6ed7c4446d","Type":"ContainerStarted","Data":"e918a9b6d6c5fbd10c042672f5b4d67b87ae3190ddaf382e145398508fbcda15"} Jul 01 18:06:36.435538 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:36.435487 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-152.ec2.internal" podStartSLOduration=4.435469809 podStartE2EDuration="4.435469809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:06:36.434656704 +0000 UTC m=+5.575840080" watchObservedRunningTime="2026-07-01 18:06:36.435469809 +0000 UTC m=+5.576653183" Jul 01 18:06:36.916678 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:36.916641 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:36.916869 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:36.916715 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:36.916869 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:36.916812 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:36.916869 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:36.916830 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:36.917012 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:36.916879 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret podName:46476d5f-12a3-4783-b432-854f32042355 nodeName:}" failed. No retries permitted until 2026-07-01 18:06:40.91685973 +0000 UTC m=+10.058043079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret") pod "global-pull-secret-syncer-5jn49" (UID: "46476d5f-12a3-4783-b432-854f32042355") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:36.917012 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:36.916899 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:06:40.916888755 +0000 UTC m=+10.058072112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:37.118782 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:37.118195 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:37.118782 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:37.118345 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:06:37.118782 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:37.118367 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:06:37.118782 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:37.118380 2572 projected.go:196] Error preparing data for projected volume kube-api-access-szjsq for pod openshift-network-diagnostics/network-check-target-sddsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:37.118782 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:37.118445 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq podName:9395f543-f981-48e3-8a19-d285b1bc933f nodeName:}" failed. No retries permitted until 2026-07-01 18:06:41.118426934 +0000 UTC m=+10.259610300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-szjsq" (UniqueName: "kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq") pod "network-check-target-sddsx" (UID: "9395f543-f981-48e3-8a19-d285b1bc933f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:37.355976 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:37.355885 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:37.356125 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:37.356092 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:37.356299 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:37.356275 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:37.356431 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:37.356399 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:38.346393 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:38.346332 2572 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0a05c0f5a3f4206ee9a1bd1853fcb5c2883aba1d3f804c8eed85a8be4ca11952: Requesting bearer token: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0a05c0f5a3f4206ee9a1bd1853fcb5c2883aba1d3f804c8eed85a8be4ca11952" Jul 01 18:06:38.346591 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:38.346463 2572 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-m2r5j_openshift-multus(c1b2d4a9-d472-4ac8-a33f-af4d676109ab): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0a05c0f5a3f4206ee9a1bd1853fcb5c2883aba1d3f804c8eed85a8be4ca11952: Requesting bearer token: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 01 18:06:38.346591 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:38.346505 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0a05c0f5a3f4206ee9a1bd1853fcb5c2883aba1d3f804c8eed85a8be4ca11952: Requesting bearer token: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="openshift-multus/multus-m2r5j" podUID="c1b2d4a9-d472-4ac8-a33f-af4d676109ab" Jul 01 18:06:38.355848 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:38.355798 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:38.356001 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:38.355935 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:38.424558 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:38.424518 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0a05c0f5a3f4206ee9a1bd1853fcb5c2883aba1d3f804c8eed85a8be4ca11952\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0a05c0f5a3f4206ee9a1bd1853fcb5c2883aba1d3f804c8eed85a8be4ca11952: Requesting bearer token: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="openshift-multus/multus-m2r5j" podUID="c1b2d4a9-d472-4ac8-a33f-af4d676109ab" Jul 01 18:06:39.355971 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:39.355899 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:39.356125 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:39.356025 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:39.356125 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:39.356089 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:39.356240 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:39.356210 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:40.355990 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:40.355952 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:40.356444 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:40.356095 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:40.945240 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:40.945204 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:40.945434 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:40.945350 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:40.945434 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:40.945377 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:40.945434 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:40.945419 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:06:48.945398422 +0000 UTC m=+18.086581785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:40.945607 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:40.945475 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:40.945607 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:40.945514 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret podName:46476d5f-12a3-4783-b432-854f32042355 nodeName:}" failed. No retries permitted until 2026-07-01 18:06:48.945503254 +0000 UTC m=+18.086686604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret") pod "global-pull-secret-syncer-5jn49" (UID: "46476d5f-12a3-4783-b432-854f32042355") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:41.147280 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:41.147244 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:41.147464 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:41.147414 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:06:41.147464 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:41.147432 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:06:41.147464 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:41.147444 2572 projected.go:196] Error preparing data for projected volume kube-api-access-szjsq for pod openshift-network-diagnostics/network-check-target-sddsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:41.147633 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:41.147498 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq podName:9395f543-f981-48e3-8a19-d285b1bc933f nodeName:}" failed. No retries permitted until 2026-07-01 18:06:49.147480299 +0000 UTC m=+18.288663653 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-szjsq" (UniqueName: "kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq") pod "network-check-target-sddsx" (UID: "9395f543-f981-48e3-8a19-d285b1bc933f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:41.356556 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:41.356484 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:41.356959 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:41.356581 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:41.356959 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:41.356644 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:41.356959 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:41.356719 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:42.355883 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:42.355852 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:42.356059 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:42.356022 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:43.356179 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:43.356138 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:43.356629 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:43.356159 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:43.356629 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:43.356294 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:43.356629 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:43.356378 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:44.355381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:44.355346 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:44.355556 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:44.355470 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:45.355588 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:45.355549 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:45.356085 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:45.355553 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:45.356085 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:45.355672 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:45.356085 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:45.355734 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:46.356023 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:46.355990 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:46.356472 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:46.356114 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:47.355653 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.355355 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:47.355653 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:47.355481 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:47.355653 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.355515 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:47.355653 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:47.355587 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:47.443715 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.442462 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" event={"ID":"9be98694-626a-44c6-af62-6c86c68a8d7a","Type":"ContainerStarted","Data":"24270d6d39f368f177a1fd222636b75fa9f08e61c9e4b1d5f09ed2ef1d43a566"} Jul 01 18:06:47.449329 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.449291 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" event={"ID":"4d53392f-dc5c-4868-8447-4a5f8c5f6f16","Type":"ContainerStarted","Data":"83ac02c453fd5befaf553e3037df16be1eb6dc8d85f7ca764ce433865f67297f"} Jul 01 18:06:47.450999 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.450926 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerStarted","Data":"5d0dd8debcad323ca649193e1ac4ee7984ad330a0672ad607113d79b9fd76bc9"} Jul 01 18:06:47.453520 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.453492 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"88726bbb6f2ec0a698ab9c9602c681c88e704e42ef2bbac9f05f3382ad2e46cb"} Jul 01 18:06:47.456802 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.455915 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f5fv7" event={"ID":"8528571c-8df9-4351-b771-3d37effe01ca","Type":"ContainerStarted","Data":"67cb3cd9335c5158faf242ffe139754a5547ee6097a8551ddef8ddbc0bfccab5"} Jul 01 18:06:47.458445 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.458414 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhlpx" event={"ID":"a9418d21-5deb-43b9-aa86-476f01d902e5","Type":"ContainerStarted","Data":"332b70d19d16cd681fb16e1aee5eec192d533088645d4429edc4bde051432618"} Jul 01 18:06:47.465387 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.465347 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pl4gm" podStartSLOduration=3.65151348 podStartE2EDuration="16.465336464s" totalImagesPullingTime="12.813822984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:06:47.464986989 +0000 UTC m=+16.606170360" watchObservedRunningTime="2026-07-01 18:06:47.465336464 +0000 UTC m=+16.606519833" Jul 01 18:06:47.498959 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.498900 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f5fv7" podStartSLOduration=3.692760202 podStartE2EDuration="16.498881193s" totalImagesPullingTime="12.806120991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:06:47.498163037 +0000 UTC m=+16.639346409" watchObservedRunningTime="2026-07-01 18:06:47.498881193 +0000 UTC m=+16.640064567" Jul 01 18:06:47.512133 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:47.512095 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vhlpx" podStartSLOduration=3.672930549 podStartE2EDuration="16.512077786s" totalImagesPullingTime="12.839147237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:06:47.511963145 +0000 UTC m=+16.653146521" watchObservedRunningTime="2026-07-01 18:06:47.512077786 +0000 UTC m=+16.653261164" Jul 01 18:06:48.145416 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.145330 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:48.146137 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.146118 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:48.355253 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.355227 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:48.355400 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:48.355340 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:48.441095 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.440956 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 18:06:48.461890 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.461866 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f9ea0bd-275c-4941-abea-106b2c01ab01" containerID="5d0dd8debcad323ca649193e1ac4ee7984ad330a0672ad607113d79b9fd76bc9" exitCode=0 Jul 01 18:06:48.462509 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.461941 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerDied","Data":"5d0dd8debcad323ca649193e1ac4ee7984ad330a0672ad607113d79b9fd76bc9"} Jul 01 18:06:48.464950 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.464932 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"bd8bccdd0b93ad430585bffd02e116d231960e9ef8bcec640be76ce7e3f067a7"} Jul 01 18:06:48.465055 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.464984 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"b430771e986a5bcc7088cf178e23ce74f78904af05198ce8d33da9cdd161c085"} Jul 01 18:06:48.465055 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.465023 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"2c2eeb4f25e7413461dc13a4c8b7afc2f7116d7bad1b28914a5dbf3c38131026"} Jul 01 18:06:48.465142 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.465059 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"329a40d8a8c34de4664e6284522dc0dd47b55552f6c62d922bea75a3fb48363d"} Jul 01 18:06:48.465142 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.465101 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"22b12d4e6883618c6c4c280ffac21191dbe4c7ce182e6c43713c3d690e080b68"} Jul 01 18:06:48.466562 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:48.466533 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" event={"ID":"9be98694-626a-44c6-af62-6c86c68a8d7a","Type":"ContainerStarted","Data":"c88c5676de1addb77519dbc861ecfd48bedfb9880a782de5ccd27e1f5e0614a3"} Jul 01 18:06:49.007637 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.007601 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:49.007843 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.007662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:49.007843 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.007764 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:49.007843 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.007781 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:49.007843 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.007846 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:07:05.007826789 +0000 UTC m=+34.149010153 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:06:49.008060 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.007863 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret podName:46476d5f-12a3-4783-b432-854f32042355 nodeName:}" failed. No retries permitted until 2026-07-01 18:07:05.007855304 +0000 UTC m=+34.149038654 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret") pod "global-pull-secret-syncer-5jn49" (UID: "46476d5f-12a3-4783-b432-854f32042355") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:06:49.208711 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.208682 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:49.208882 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.208861 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:06:49.208926 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.208888 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:06:49.208926 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.208901 2572 projected.go:196] Error preparing data for projected volume kube-api-access-szjsq for pod openshift-network-diagnostics/network-check-target-sddsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:49.209013 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.208963 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq podName:9395f543-f981-48e3-8a19-d285b1bc933f nodeName:}" failed. No retries permitted until 2026-07-01 18:07:05.208940885 +0000 UTC m=+34.350124258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-szjsq" (UniqueName: "kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq") pod "network-check-target-sddsx" (UID: "9395f543-f981-48e3-8a19-d285b1bc933f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:06:49.348311 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.348159 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T18:06:48.441091226Z","UUID":"ffec4201-6f31-4896-93d8-3b8a03a8f8e3","Handler":null,"Name":"","Endpoint":""} Jul 01 18:06:49.349842 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.349818 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 18:06:49.349963 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.349864 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 18:06:49.356179 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.356155 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:49.356310 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.356192 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:49.356409 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.356380 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:49.356681 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:49.356652 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:49.470019 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.469980 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mrltk" event={"ID":"250e8f25-8966-4f14-83ab-1e5b42aeb73a","Type":"ContainerStarted","Data":"bd1bfd8485564ce34952e96a7f6718af3779fb771025382b3d9983acf7c4bb2a"} Jul 01 18:06:49.472054 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.472020 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" event={"ID":"9be98694-626a-44c6-af62-6c86c68a8d7a","Type":"ContainerStarted","Data":"8f844e93ba2d92021c37210106c14bf29073f849571e5f2f1553f70c2042e06e"} Jul 01 18:06:49.472054 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.472044 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 18:06:49.484125 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.484081 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mrltk" podStartSLOduration=5.675901127 podStartE2EDuration="18.484067727s" totalImagesPullingTime="12.8081666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:06:49.483601708 +0000 UTC m=+18.624785079" watchObservedRunningTime="2026-07-01 18:06:49.484067727 +0000 UTC m=+18.625251098" Jul 01 18:06:49.500783 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.500734 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fccpl" podStartSLOduration=4.032910533 podStartE2EDuration="18.500720667s" totalImagesPullingTime="14.467810134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:06:49.500413165 +0000 UTC m=+18.641596537" watchObservedRunningTime="2026-07-01 18:06:49.500720667 +0000 UTC m=+18.641904038" Jul 01 18:06:49.764528 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.764419 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:49.764993 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:49.764970 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vhlpx" Jul 01 18:06:50.356007 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:50.355974 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:50.356253 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:50.356217 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:50.478649 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:50.478612 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"2fe3bd23c25e2fa5c0afea2961a740d0510132f4081d7ac637f68c2ec11b0cc9"} Jul 01 18:06:51.358397 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:51.356304 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:51.358397 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:51.356424 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:51.358397 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:51.357969 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:51.358397 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:51.358058 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:52.355307 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:52.355235 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:52.355733 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:52.355362 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:53.355720 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:53.355680 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:53.356197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:53.355727 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:53.356197 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:53.355823 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:53.356197 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:53.355933 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:53.495965 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:53.495925 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhlpx_a9418d21-5deb-43b9-aa86-476f01d902e5/konnectivity-agent/0.log" limit=1024 Jul 01 18:06:53.576490 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:53.576445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhlpx_a9418d21-5deb-43b9-aa86-476f01d902e5/konnectivity-agent/0.log" limit=1024 Jul 01 18:06:54.355829 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.355586 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:54.356469 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:54.355898 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:54.491224 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.491184 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lnrs5" event={"ID":"9de04a82-ce0c-4dfa-b8e1-1db529bdfbab","Type":"ContainerStarted","Data":"e4170778d6f597d924fb97a57b48f6b89f8a7d6e3679c0117eace626bc1839cf"} Jul 01 18:06:54.493341 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.493304 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f9ea0bd-275c-4941-abea-106b2c01ab01" containerID="35fc619029180c1ec3b30cd0ff82a00552c8b5e8df46e6a8257b553e35be53de" exitCode=0 Jul 01 18:06:54.493471 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.493390 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerDied","Data":"35fc619029180c1ec3b30cd0ff82a00552c8b5e8df46e6a8257b553e35be53de"} Jul 01 18:06:54.498081 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.498059 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" event={"ID":"627519ad-b343-43ab-8f0f-9fc101b7cb91","Type":"ContainerStarted","Data":"256e2f44d39fbf373db524faa3a1f39189c45e4345185ff471e62eb78b63db1b"} Jul 01 18:06:54.498406 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.498385 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:54.498498 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.498459 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:54.498549 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.498504 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:54.505878 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.505748 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lnrs5" podStartSLOduration=3.975622355 podStartE2EDuration="23.505731642s" totalImagesPullingTime="19.530109287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-01 18:06:54.505494367 +0000 UTC m=+23.646677739" watchObservedRunningTime="2026-07-01 18:06:54.505731642 +0000 UTC m=+23.646915015" Jul 01 18:06:54.515531 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.515508 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:54.515650 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.515638 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:06:54.550539 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:54.550499 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" podStartSLOduration=10.609052111 podStartE2EDuration="23.550482701s" totalImagesPullingTime="12.94143059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:06:54.550232947 +0000 UTC m=+23.691416319" watchObservedRunningTime="2026-07-01 18:06:54.550482701 +0000 UTC m=+23.691666074" Jul 01 18:06:55.360138 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:55.360114 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:55.360609 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:55.360141 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:55.360609 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:55.360236 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:55.360609 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:55.360336 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:55.502702 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:55.502667 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f9ea0bd-275c-4941-abea-106b2c01ab01" containerID="aa7e09d01fb3ae7bcb85b062422ff893879a0f860afb5a17a49a9a934af90804" exitCode=0 Jul 01 18:06:55.502840 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:55.502765 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerDied","Data":"aa7e09d01fb3ae7bcb85b062422ff893879a0f860afb5a17a49a9a934af90804"} Jul 01 18:06:56.033067 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:56.033031 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dr9zg"] Jul 01 18:06:56.033253 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:56.033178 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:56.033318 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:56.033299 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:56.036502 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:56.036470 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5jn49"] Jul 01 18:06:56.036941 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:56.036566 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sddsx"] Jul 01 18:06:56.036941 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:56.036586 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:56.036941 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:56.036691 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:56.036941 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:56.036739 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:56.036941 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:56.036826 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:57.355754 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:57.355717 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:57.356222 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:57.355890 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:06:58.355536 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:58.355459 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:06:58.355536 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:58.355503 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:06:58.355786 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:58.355595 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:06:58.355786 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:58.355736 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:06:59.356210 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:06:59.356140 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:06:59.356764 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:06:59.356274 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:07:00.355391 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:00.355362 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:07:00.355592 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:00.355362 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:07:00.355592 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:00.355502 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jn49" podUID="46476d5f-12a3-4783-b432-854f32042355" Jul 01 18:07:00.355592 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:00.355529 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sddsx" podUID="9395f543-f981-48e3-8a19-d285b1bc933f" Jul 01 18:07:00.517720 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:00.517643 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m2r5j" event={"ID":"c1b2d4a9-d472-4ac8-a33f-af4d676109ab","Type":"ContainerStarted","Data":"7a39cb8a0f2e2a984807e387dba997bd6afabe26f7dcfbdb6eba549b72b4ff7a"} Jul 01 18:07:00.534937 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:00.534884 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-m2r5j" podStartSLOduration=4.012270568 podStartE2EDuration="29.534868663s" totalImagesPullingTime="25.522598095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-01 18:07:00.534594553 +0000 UTC m=+29.675777925" watchObservedRunningTime="2026-07-01 18:07:00.534868663 +0000 UTC m=+29.676052035" Jul 01 18:07:01.356884 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.356692 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:07:01.357003 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:01.356962 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:07:01.522105 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.522072 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f9ea0bd-275c-4941-abea-106b2c01ab01" containerID="1c1eea12f01e451240c9ccdc91d8f5cae3939ed991455fdc8f1adbdb3201ba7b" exitCode=0 Jul 01 18:07:01.522571 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.522124 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerDied","Data":"1c1eea12f01e451240c9ccdc91d8f5cae3939ed991455fdc8f1adbdb3201ba7b"} Jul 01 18:07:01.578397 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.578340 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-152.ec2.internal" event="NodeReady" Jul 01 18:07:01.578491 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.578444 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 18:07:01.620006 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.619981 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wmv7g"] Jul 01 18:07:01.635298 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.635280 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.635407 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.635299 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zbkc9"] Jul 01 18:07:01.637810 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.637792 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 18:07:01.637922 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.637826 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 18:07:01.638556 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.638542 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vsbfc\"" Jul 01 18:07:01.653628 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.653610 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wmv7g"] Jul 01 18:07:01.653723 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.653637 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zbkc9"] Jul 01 18:07:01.653723 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.653707 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zbkc9" Jul 01 18:07:01.656009 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.655991 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 18:07:01.656009 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.656008 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cl7bm\"" Jul 01 18:07:01.656167 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.656011 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 18:07:01.656167 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.656052 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 18:07:01.706957 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.706939 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.707077 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.706968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c83a996-6382-4768-ba63-7d3a5067953a-tmp-dir\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.707077 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.707008 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2c83a996-6382-4768-ba63-7d3a5067953a-config-volume\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.707077 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.707027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptb7f\" (UniqueName: \"kubernetes.io/projected/2c83a996-6382-4768-ba63-7d3a5067953a-kube-api-access-ptb7f\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.808097 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.808071 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.808202 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.808110 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c83a996-6382-4768-ba63-7d3a5067953a-tmp-dir\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.808202 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.808140 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6c1a534-badc-41f6-95a8-2ebcba6394fc-cert\") pod \"ingress-canary-zbkc9\" (UID: \"c6c1a534-badc-41f6-95a8-2ebcba6394fc\") " pod="openshift-ingress-canary/ingress-canary-zbkc9" Jul 01 18:07:01.808297 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:01.808224 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:07:01.808297 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.808277 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2c83a996-6382-4768-ba63-7d3a5067953a-config-volume\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.808297 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:01.808292 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:07:02.308273171 +0000 UTC m=+31.449456539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:07:01.808447 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.808333 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptb7f\" (UniqueName: \"kubernetes.io/projected/2c83a996-6382-4768-ba63-7d3a5067953a-kube-api-access-ptb7f\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.808447 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.808362 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxw4h\" (UniqueName: \"kubernetes.io/projected/c6c1a534-badc-41f6-95a8-2ebcba6394fc-kube-api-access-rxw4h\") pod \"ingress-canary-zbkc9\" (UID: \"c6c1a534-badc-41f6-95a8-2ebcba6394fc\") " pod="openshift-ingress-canary/ingress-canary-zbkc9" Jul 01 18:07:01.808447 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.808443 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c83a996-6382-4768-ba63-7d3a5067953a-tmp-dir\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.808754 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.808734 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2c83a996-6382-4768-ba63-7d3a5067953a-config-volume\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.819865 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.819847 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptb7f\" (UniqueName: \"kubernetes.io/projected/2c83a996-6382-4768-ba63-7d3a5067953a-kube-api-access-ptb7f\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:01.909107 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.909089 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6c1a534-badc-41f6-95a8-2ebcba6394fc-cert\") pod \"ingress-canary-zbkc9\" (UID: \"c6c1a534-badc-41f6-95a8-2ebcba6394fc\") " pod="openshift-ingress-canary/ingress-canary-zbkc9" Jul 01 18:07:01.909199 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.909138 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxw4h\" (UniqueName: \"kubernetes.io/projected/c6c1a534-badc-41f6-95a8-2ebcba6394fc-kube-api-access-rxw4h\") pod \"ingress-canary-zbkc9\" (UID: \"c6c1a534-badc-41f6-95a8-2ebcba6394fc\") " pod="openshift-ingress-canary/ingress-canary-zbkc9" Jul 01 18:07:01.911261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.911245 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6c1a534-badc-41f6-95a8-2ebcba6394fc-cert\") pod \"ingress-canary-zbkc9\" (UID: \"c6c1a534-badc-41f6-95a8-2ebcba6394fc\") " pod="openshift-ingress-canary/ingress-canary-zbkc9" Jul 01 18:07:01.916526 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.916509 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxw4h\" (UniqueName: \"kubernetes.io/projected/c6c1a534-badc-41f6-95a8-2ebcba6394fc-kube-api-access-rxw4h\") pod \"ingress-canary-zbkc9\" (UID: \"c6c1a534-badc-41f6-95a8-2ebcba6394fc\") " pod="openshift-ingress-canary/ingress-canary-zbkc9" Jul 01 18:07:01.960818 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:01.960796 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zbkc9" Jul 01 18:07:02.153593 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.153560 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zbkc9"] Jul 01 18:07:02.157974 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:07:02.157763 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c1a534_badc_41f6_95a8_2ebcba6394fc.slice/crio-57a5ba6a5ce2b8bc4c0a980954afdc87dd722e5d5d81a13738f90bd0bfab9391 WatchSource:0}: Error finding container 57a5ba6a5ce2b8bc4c0a980954afdc87dd722e5d5d81a13738f90bd0bfab9391: Status 404 returned error can't find the container with id 57a5ba6a5ce2b8bc4c0a980954afdc87dd722e5d5d81a13738f90bd0bfab9391 Jul 01 18:07:02.312064 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.312034 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:02.312211 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:02.312195 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:07:02.312277 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:02.312266 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:07:03.312246493 +0000 UTC m=+32.453429844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:07:02.356238 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.356210 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:07:02.356238 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.356239 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:07:02.358862 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.358832 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 18:07:02.358862 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.358834 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 18:07:02.359036 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.358834 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 18:07:02.359036 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.358899 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pdj6p\"" Jul 01 18:07:02.526209 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:02.526117 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zbkc9" event={"ID":"c6c1a534-badc-41f6-95a8-2ebcba6394fc","Type":"ContainerStarted","Data":"57a5ba6a5ce2b8bc4c0a980954afdc87dd722e5d5d81a13738f90bd0bfab9391"} Jul 01 18:07:03.320261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:03.320221 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:03.320446 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:03.320389 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:07:03.320518 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:03.320467 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:07:05.320445467 +0000 UTC m=+34.461628817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:07:03.355633 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:03.355601 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:07:03.358145 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:03.358123 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 18:07:03.358279 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:03.358126 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bhlcl\"" Jul 01 18:07:05.032142 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:05.032105 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:07:05.032581 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:05.032155 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:07:05.032581 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:05.032271 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 18:07:05.032581 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:05.032355 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:07:37.032335242 +0000 UTC m=+66.173518609 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : secret "metrics-daemon-secret" not found Jul 01 18:07:05.035506 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:05.035482 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46476d5f-12a3-4783-b432-854f32042355-original-pull-secret\") pod \"global-pull-secret-syncer-5jn49\" (UID: \"46476d5f-12a3-4783-b432-854f32042355\") " pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:07:05.074563 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:05.074525 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jn49" Jul 01 18:07:05.234286 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:05.234248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:07:05.237158 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:05.237105 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szjsq\" (UniqueName: \"kubernetes.io/projected/9395f543-f981-48e3-8a19-d285b1bc933f-kube-api-access-szjsq\") pod \"network-check-target-sddsx\" (UID: \"9395f543-f981-48e3-8a19-d285b1bc933f\") " pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:07:05.334918 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:05.334831 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:05.335084 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:05.334974 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:07:05.335084 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:05.335029 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:07:09.335016089 +0000 UTC m=+38.476199439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:07:05.369120 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:05.369087 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:07:07.050235 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:07.050201 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sddsx"] Jul 01 18:07:07.054686 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:07:07.054659 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9395f543_f981_48e3_8a19_d285b1bc933f.slice/crio-b6b373252b0fba61753d4b8a4c34974950b81cccb87e3220954604e26d0b2d9e WatchSource:0}: Error finding container b6b373252b0fba61753d4b8a4c34974950b81cccb87e3220954604e26d0b2d9e: Status 404 returned error can't find the container with id b6b373252b0fba61753d4b8a4c34974950b81cccb87e3220954604e26d0b2d9e Jul 01 18:07:07.065700 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:07.065674 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5jn49"] Jul 01 18:07:07.067897 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:07:07.067863 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46476d5f_12a3_4783_b432_854f32042355.slice/crio-68dda5086c9a7b0b4f6e4ce7a24e7eb5c678c691a996f9bf00b0e5c9c1480775 WatchSource:0}: Error finding container 68dda5086c9a7b0b4f6e4ce7a24e7eb5c678c691a996f9bf00b0e5c9c1480775: Status 404 returned error can't find the container with id 68dda5086c9a7b0b4f6e4ce7a24e7eb5c678c691a996f9bf00b0e5c9c1480775 Jul 01 18:07:07.539097 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:07.539018 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sddsx" event={"ID":"9395f543-f981-48e3-8a19-d285b1bc933f","Type":"ContainerStarted","Data":"b6b373252b0fba61753d4b8a4c34974950b81cccb87e3220954604e26d0b2d9e"} Jul 01 18:07:07.540434 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:07.540409 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zbkc9" event={"ID":"c6c1a534-badc-41f6-95a8-2ebcba6394fc","Type":"ContainerStarted","Data":"6a9934746e01a71ea2d0b241aa0a26476175fd264cfa1563e3ccb9b6577c0951"} Jul 01 18:07:07.541496 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:07.541471 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5jn49" event={"ID":"46476d5f-12a3-4783-b432-854f32042355","Type":"ContainerStarted","Data":"68dda5086c9a7b0b4f6e4ce7a24e7eb5c678c691a996f9bf00b0e5c9c1480775"} Jul 01 18:07:07.544088 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:07.544063 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f9ea0bd-275c-4941-abea-106b2c01ab01" containerID="7f81be13dd7f9e2ef95229ff5f2ffc8051786def0bf3734a391ef043694588c0" exitCode=0 Jul 01 18:07:07.544173 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:07.544102 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerDied","Data":"7f81be13dd7f9e2ef95229ff5f2ffc8051786def0bf3734a391ef043694588c0"} Jul 01 18:07:07.556635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:07.556599 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zbkc9" podStartSLOduration=1.823414354 podStartE2EDuration="6.556585882s" totalImagesPullingTime="4.733171528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:07:07.555163732 +0000 UTC m=+36.696347103" watchObservedRunningTime="2026-07-01 18:07:07.556585882 +0000 UTC m=+36.697769254" Jul 01 18:07:08.550435 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:08.550394 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f9ea0bd-275c-4941-abea-106b2c01ab01" containerID="81d52fb39dcb95eb7ef1eb87ef089d0f505b4be37ae02b21aa6e21c39edf6afd" exitCode=0 Jul 01 18:07:08.550845 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:08.550479 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerDied","Data":"81d52fb39dcb95eb7ef1eb87ef089d0f505b4be37ae02b21aa6e21c39edf6afd"} Jul 01 18:07:09.365887 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:09.365854 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:09.366081 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:09.366013 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:07:09.366149 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:09.366088 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:07:17.366066537 +0000 UTC m=+46.507249887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:07:09.557257 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:09.557209 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sftwb" event={"ID":"5f9ea0bd-275c-4941-abea-106b2c01ab01","Type":"ContainerStarted","Data":"592973e658c9c7250ffd48257f3b95c4eb8beda5ab6c9dfb9ffb04cfeb873953"} Jul 01 18:07:09.581250 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:09.581192 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sftwb" podStartSLOduration=8.71008186 podStartE2EDuration="38.581158323s" totalImagesPullingTime="29.871076463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:07:09.579837809 +0000 UTC m=+38.721021180" watchObservedRunningTime="2026-07-01 18:07:09.581158323 +0000 UTC m=+38.722341699" Jul 01 18:07:10.561744 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:10.561652 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sddsx" event={"ID":"9395f543-f981-48e3-8a19-d285b1bc933f","Type":"ContainerStarted","Data":"f4a1fcb40d7751e4e652f847b341c5c8857eb29f9c2948db99a7a710d198bf37"} Jul 01 18:07:10.562204 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:10.561888 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:07:10.577173 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:10.577121 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sddsx" podStartSLOduration=36.332958769 podStartE2EDuration="39.57710405s" totalImagesPullingTime="3.244145281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:07:10.576224261 +0000 UTC m=+39.717407645" watchObservedRunningTime="2026-07-01 18:07:10.57710405 +0000 UTC m=+39.718287422" Jul 01 18:07:14.573636 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:14.573597 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5jn49" event={"ID":"46476d5f-12a3-4783-b432-854f32042355","Type":"ContainerStarted","Data":"a793b06290393ded703052eba77ce84a52045eaaee798f654472bc9b20ca487c"} Jul 01 18:07:14.587784 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:14.587723 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5jn49" podStartSLOduration=36.659933574 podStartE2EDuration="43.587710426s" totalImagesPullingTime="6.927776852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:07:14.587508224 +0000 UTC m=+43.728691608" watchObservedRunningTime="2026-07-01 18:07:14.587710426 +0000 UTC m=+43.728893797" Jul 01 18:07:17.423325 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:17.423289 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:17.423710 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:17.423386 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:07:17.423710 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:17.423436 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:07:33.423423127 +0000 UTC m=+62.564606477 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:07:26.524912 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:26.524880 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xmjfl" Jul 01 18:07:33.431105 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:33.431070 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:07:33.431493 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:33.431180 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:07:33.431493 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:33.431230 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:08:05.431216575 +0000 UTC m=+94.572399925 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:07:37.054976 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:37.054941 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:07:37.055404 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:37.055060 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 18:07:37.055404 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:07:37.055113 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:08:41.055100108 +0000 UTC m=+130.196283458 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : secret "metrics-daemon-secret" not found Jul 01 18:07:41.566886 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:07:41.566856 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sddsx" Jul 01 18:08:05.446318 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:08:05.446269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:08:05.446734 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:08:05.446417 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:08:05.446734 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:08:05.446478 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:09:09.446462568 +0000 UTC m=+158.587645918 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:08:41.073343 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:08:41.073301 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:08:41.073849 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:08:41.073461 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 18:08:41.073849 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:08:41.073528 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs podName:d7e0bb01-e2e5-41b6-a552-ae7b6380eaec nodeName:}" failed. No retries permitted until 2026-07-01 18:10:43.073513152 +0000 UTC m=+252.214696502 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs") pod "network-metrics-daemon-dr9zg" (UID: "d7e0bb01-e2e5-41b6-a552-ae7b6380eaec") : secret "metrics-daemon-secret" not found Jul 01 18:09:04.644787 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:09:04.644731 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wmv7g" podUID="2c83a996-6382-4768-ba63-7d3a5067953a" Jul 01 18:09:04.824407 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:09:04.824380 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wmv7g" Jul 01 18:09:06.371666 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:09:06.371634 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dr9zg" podUID="d7e0bb01-e2e5-41b6-a552-ae7b6380eaec" Jul 01 18:09:09.456535 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:09:09.456477 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:09:09.456967 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:09:09.456600 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:09:09.456967 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:09:09.456670 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls podName:2c83a996-6382-4768-ba63-7d3a5067953a nodeName:}" failed. No retries permitted until 2026-07-01 18:11:11.456653104 +0000 UTC m=+280.597836459 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls") pod "dns-default-wmv7g" (UID: "2c83a996-6382-4768-ba63-7d3a5067953a") : secret "dns-default-metrics-tls" not found Jul 01 18:09:21.356451 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:09:21.356415 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:10:03.501192 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.501155 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw"] Jul 01 18:10:03.503079 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.503059 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.508354 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.508326 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 18:10:03.508354 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.508334 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ctw79\"" Jul 01 18:10:03.508926 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.508903 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 18:10:03.510039 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.510006 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 01 18:10:03.510146 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.510101 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 01 18:10:03.510892 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.510872 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-86dc5bb4b9-fkb96"] Jul 01 18:10:03.512839 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.512815 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.518999 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.518978 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 01 18:10:03.519107 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.519041 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 01 18:10:03.519107 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.519050 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 01 18:10:03.519107 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.519063 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 01 18:10:03.519107 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.519086 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 01 18:10:03.519406 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.519293 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2d7br\"" Jul 01 18:10:03.519837 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.519822 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 01 18:10:03.528916 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.528898 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw"] Jul 01 18:10:03.534404 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.534384 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86dc5bb4b9-fkb96"] Jul 01 18:10:03.593547 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.593523 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz"] Jul 01 18:10:03.595479 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.595461 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.597918 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.597899 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 01 18:10:03.598027 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.597978 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 01 18:10:03.598027 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.598005 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:10:03.598180 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.598162 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh"] Jul 01 18:10:03.598266 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.598165 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mgvk4\"" Jul 01 18:10:03.598387 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.598372 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 01 18:10:03.599951 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.599925 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:03.602122 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.602109 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vsj8r\"" Jul 01 18:10:03.602512 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.602348 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 01 18:10:03.602512 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.602453 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:10:03.602512 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.602491 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 01 18:10:03.613498 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.613479 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz"] Jul 01 18:10:03.615974 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.615958 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/933d818a-0783-4070-9394-957b53f50751-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.616040 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.615982 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-stats-auth\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.616040 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.616006 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.616119 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.616086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.616119 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.616108 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc2dw\" (UniqueName: \"kubernetes.io/projected/3c23be7d-3674-4403-b8ba-31854ca8de3c-kube-api-access-mc2dw\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.616184 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.616131 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5h62\" (UniqueName: \"kubernetes.io/projected/933d818a-0783-4070-9394-957b53f50751-kube-api-access-k5h62\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.616184 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.616153 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.616254 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.616209 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-default-certificate\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.618101 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.618080 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh"] Jul 01 18:10:03.716618 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.716586 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/933d818a-0783-4070-9394-957b53f50751-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.716618 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.716613 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-stats-auth\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.716830 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.716636 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.716830 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.716679 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.716830 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:03.716794 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 18:10:03.716943 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:03.716802 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:04.216765395 +0000 UTC m=+213.357948746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : configmap references non-existent config key: service-ca.crt Jul 01 18:10:03.716943 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.716862 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2dw\" (UniqueName: \"kubernetes.io/projected/3c23be7d-3674-4403-b8ba-31854ca8de3c-kube-api-access-mc2dw\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.716943 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:03.716899 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:04.216886872 +0000 UTC m=+213.358070226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : secret "router-metrics-certs-default" not found Jul 01 18:10:03.717136 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.716977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5h62\" (UniqueName: \"kubernetes.io/projected/933d818a-0783-4070-9394-957b53f50751-kube-api-access-k5h62\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.717136 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.716998 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2af814a3-00a8-4ace-97b7-5c069a6cede2-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.717136 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.717021 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:03.717136 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.717053 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.717136 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.717078 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2af814a3-00a8-4ace-97b7-5c069a6cede2-config\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.717376 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:03.717138 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:03.717376 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.717163 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltt25\" (UniqueName: \"kubernetes.io/projected/2af814a3-00a8-4ace-97b7-5c069a6cede2-kube-api-access-ltt25\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.717376 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:03.717202 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls podName:933d818a-0783-4070-9394-957b53f50751 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:04.217186066 +0000 UTC m=+213.358369430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t7phw" (UID: "933d818a-0783-4070-9394-957b53f50751") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:03.717376 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.717227 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrxr6\" (UniqueName: \"kubernetes.io/projected/dfe2d982-946e-45e1-9cac-f3072e7c8917-kube-api-access-qrxr6\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:03.717376 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.717291 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-default-certificate\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.717376 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.717337 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/933d818a-0783-4070-9394-957b53f50751-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.719067 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.719049 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-stats-auth\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.719417 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.719398 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-default-certificate\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.725465 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.725443 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5h62\" (UniqueName: \"kubernetes.io/projected/933d818a-0783-4070-9394-957b53f50751-kube-api-access-k5h62\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:03.726858 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.726837 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc2dw\" (UniqueName: \"kubernetes.io/projected/3c23be7d-3674-4403-b8ba-31854ca8de3c-kube-api-access-mc2dw\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:03.818285 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.818213 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2af814a3-00a8-4ace-97b7-5c069a6cede2-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.818285 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.818242 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:03.818285 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.818267 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2af814a3-00a8-4ace-97b7-5c069a6cede2-config\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.818285 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.818284 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltt25\" (UniqueName: \"kubernetes.io/projected/2af814a3-00a8-4ace-97b7-5c069a6cede2-kube-api-access-ltt25\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.818601 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.818300 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrxr6\" (UniqueName: \"kubernetes.io/projected/dfe2d982-946e-45e1-9cac-f3072e7c8917-kube-api-access-qrxr6\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:03.818601 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:03.818361 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 18:10:03.818601 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:03.818430 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls podName:dfe2d982-946e-45e1-9cac-f3072e7c8917 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:04.31841288 +0000 UTC m=+213.459596230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-29swh" (UID: "dfe2d982-946e-45e1-9cac-f3072e7c8917") : secret "samples-operator-tls" not found Jul 01 18:10:03.818936 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.818914 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2af814a3-00a8-4ace-97b7-5c069a6cede2-config\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.820380 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.820364 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2af814a3-00a8-4ace-97b7-5c069a6cede2-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.829254 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.829229 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltt25\" (UniqueName: \"kubernetes.io/projected/2af814a3-00a8-4ace-97b7-5c069a6cede2-kube-api-access-ltt25\") pod \"service-ca-operator-5d7b7dc95c-qgjgz\" (UID: \"2af814a3-00a8-4ace-97b7-5c069a6cede2\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:03.829414 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.829397 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrxr6\" (UniqueName: \"kubernetes.io/projected/dfe2d982-946e-45e1-9cac-f3072e7c8917-kube-api-access-qrxr6\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:03.906384 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:03.906347 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" Jul 01 18:10:04.021288 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:04.021262 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz"] Jul 01 18:10:04.024485 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:04.024461 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af814a3_00a8_4ace_97b7_5c069a6cede2.slice/crio-c957ae19f4dc52b56883699282eccdc664f1879b8a8664e3d74f9c44cf1bb09e WatchSource:0}: Error finding container c957ae19f4dc52b56883699282eccdc664f1879b8a8664e3d74f9c44cf1bb09e: Status 404 returned error can't find the container with id c957ae19f4dc52b56883699282eccdc664f1879b8a8664e3d74f9c44cf1bb09e Jul 01 18:10:04.221955 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:04.221926 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:04.222102 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:04.221977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:04.222102 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:04.222039 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:04.222102 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:04.222096 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:05.222078766 +0000 UTC m=+214.363262130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : configmap references non-existent config key: service-ca.crt Jul 01 18:10:04.222208 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:04.222150 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:04.222208 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:04.222200 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 18:10:04.222264 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:04.222247 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls podName:933d818a-0783-4070-9394-957b53f50751 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:05.222231355 +0000 UTC m=+214.363414708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t7phw" (UID: "933d818a-0783-4070-9394-957b53f50751") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:04.222264 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:04.222262 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:05.222256507 +0000 UTC m=+214.363439856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : secret "router-metrics-certs-default" not found Jul 01 18:10:04.322577 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:04.322543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:04.322716 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:04.322677 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 18:10:04.322804 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:04.322733 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls podName:dfe2d982-946e-45e1-9cac-f3072e7c8917 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:05.322719111 +0000 UTC m=+214.463902461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-29swh" (UID: "dfe2d982-946e-45e1-9cac-f3072e7c8917") : secret "samples-operator-tls" not found Jul 01 18:10:04.963165 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:04.963133 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" event={"ID":"2af814a3-00a8-4ace-97b7-5c069a6cede2","Type":"ContainerStarted","Data":"c957ae19f4dc52b56883699282eccdc664f1879b8a8664e3d74f9c44cf1bb09e"} Jul 01 18:10:05.229036 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:05.228915 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:05.229036 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:05.228971 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:05.229036 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:05.229002 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:05.229251 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:05.229052 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 18:10:05.229251 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:05.229080 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:05.229251 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:05.229115 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:07.229100927 +0000 UTC m=+216.370284281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : secret "router-metrics-certs-default" not found Jul 01 18:10:05.229251 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:05.229130 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls podName:933d818a-0783-4070-9394-957b53f50751 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:07.229123093 +0000 UTC m=+216.370306443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t7phw" (UID: "933d818a-0783-4070-9394-957b53f50751") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:05.229251 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:05.229142 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:07.229134539 +0000 UTC m=+216.370317888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : configmap references non-existent config key: service-ca.crt Jul 01 18:10:05.329324 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:05.329294 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:05.329491 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:05.329422 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 18:10:05.329491 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:05.329474 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls podName:dfe2d982-946e-45e1-9cac-f3072e7c8917 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:07.329460426 +0000 UTC m=+216.470643781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-29swh" (UID: "dfe2d982-946e-45e1-9cac-f3072e7c8917") : secret "samples-operator-tls" not found Jul 01 18:10:07.247390 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:07.247357 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:07.247828 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:07.247413 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:07.247828 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:07.247537 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 18:10:07.247828 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:07.247547 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:11.247527493 +0000 UTC m=+220.388710866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : configmap references non-existent config key: service-ca.crt Jul 01 18:10:07.247828 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:07.247634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:07.247828 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:07.247687 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:07.247828 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:07.247716 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:11.247702457 +0000 UTC m=+220.388885807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : secret "router-metrics-certs-default" not found Jul 01 18:10:07.247828 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:07.247739 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls podName:933d818a-0783-4070-9394-957b53f50751 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:11.247728055 +0000 UTC m=+220.388911404 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t7phw" (UID: "933d818a-0783-4070-9394-957b53f50751") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:07.348145 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:07.348111 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:07.348313 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:07.348274 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 18:10:07.348376 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:07.348350 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls podName:dfe2d982-946e-45e1-9cac-f3072e7c8917 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:11.34832942 +0000 UTC m=+220.489512791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-29swh" (UID: "dfe2d982-946e-45e1-9cac-f3072e7c8917") : secret "samples-operator-tls" not found Jul 01 18:10:07.972246 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:07.972209 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" event={"ID":"2af814a3-00a8-4ace-97b7-5c069a6cede2","Type":"ContainerStarted","Data":"cc9685dde56c0766139d4919b65cb29aa377fd5426d6bc1472f7b26b4c677dfa"} Jul 01 18:10:07.989231 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:07.989183 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" podStartSLOduration=1.747722258 podStartE2EDuration="4.989163965s" totalImagesPullingTime="3.241441707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:07.988352932 +0000 UTC m=+217.129536305" watchObservedRunningTime="2026-07-01 18:10:07.989163965 +0000 UTC m=+217.130347342" Jul 01 18:10:09.841637 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.841602 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mwwtp"] Jul 01 18:10:09.843738 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.843716 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:09.846026 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.846008 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 18:10:09.846300 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.846277 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 18:10:09.847088 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.847069 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9p988\"" Jul 01 18:10:09.847148 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.847102 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 18:10:09.847202 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.847102 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 18:10:09.853945 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.853923 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mwwtp"] Jul 01 18:10:09.967892 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.967856 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44927caa-061c-4efe-a8e3-4ff490799539-crio-socket\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:09.967892 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.967891 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44927caa-061c-4efe-a8e3-4ff490799539-data-volume\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:09.968114 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.967928 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:09.968114 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.968026 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:09.968114 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:09.968059 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pq7z\" (UniqueName: \"kubernetes.io/projected/44927caa-061c-4efe-a8e3-4ff490799539-kube-api-access-2pq7z\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.068960 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.068932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.068960 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.068963 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pq7z\" (UniqueName: \"kubernetes.io/projected/44927caa-061c-4efe-a8e3-4ff490799539-kube-api-access-2pq7z\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.069186 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.069012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44927caa-061c-4efe-a8e3-4ff490799539-crio-socket\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.069186 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.069029 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44927caa-061c-4efe-a8e3-4ff490799539-data-volume\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.069186 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.069066 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.069186 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:10.069067 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:10:10.069186 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.069139 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44927caa-061c-4efe-a8e3-4ff490799539-crio-socket\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.069186 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:10.069144 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:10.569126073 +0000 UTC m=+219.710309436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : secret "insights-runtime-extractor-tls" not found Jul 01 18:10:10.069186 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:10.069151 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:10:10.069423 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:10.069208 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:10.569194561 +0000 UTC m=+219.710377914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : configmap "kube-rbac-proxy" not found Jul 01 18:10:10.069423 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.069404 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44927caa-061c-4efe-a8e3-4ff490799539-data-volume\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.084874 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.084846 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pq7z\" (UniqueName: \"kubernetes.io/projected/44927caa-061c-4efe-a8e3-4ff490799539-kube-api-access-2pq7z\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.112540 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.112467 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6"] Jul 01 18:10:10.114711 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.114688 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" Jul 01 18:10:10.117264 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.117244 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 01 18:10:10.117635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.117619 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 01 18:10:10.117710 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.117623 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kbvgw\"" Jul 01 18:10:10.125703 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.125675 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6"] Jul 01 18:10:10.169830 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.169796 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvlcc\" (UniqueName: \"kubernetes.io/projected/bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e-kube-api-access-fvlcc\") pod \"migrator-786f9fbc5-hc7x6\" (UID: \"bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" Jul 01 18:10:10.270249 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.270212 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvlcc\" (UniqueName: \"kubernetes.io/projected/bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e-kube-api-access-fvlcc\") pod \"migrator-786f9fbc5-hc7x6\" (UID: \"bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" Jul 01 18:10:10.278134 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.278102 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvlcc\" (UniqueName: \"kubernetes.io/projected/bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e-kube-api-access-fvlcc\") pod \"migrator-786f9fbc5-hc7x6\" (UID: \"bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" Jul 01 18:10:10.423708 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.423671 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" Jul 01 18:10:10.546141 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.546090 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6"] Jul 01 18:10:10.548562 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:10.548533 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6ac817_f50c_4bd3_aa74_a28ec6d34f4e.slice/crio-f591276444b4ece7fad1cb8eea8389a4101470126baa39c14152382e48090d93 WatchSource:0}: Error finding container f591276444b4ece7fad1cb8eea8389a4101470126baa39c14152382e48090d93: Status 404 returned error can't find the container with id f591276444b4ece7fad1cb8eea8389a4101470126baa39c14152382e48090d93 Jul 01 18:10:10.574025 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.573999 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.574135 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.574090 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:10.574194 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:10.574155 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:10:10.574194 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:10.574163 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:10:10.574285 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:10.574207 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:11.574195191 +0000 UTC m=+220.715378541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : configmap "kube-rbac-proxy" not found Jul 01 18:10:10.574285 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:10.574220 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:11.57421396 +0000 UTC m=+220.715397309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : secret "insights-runtime-extractor-tls" not found Jul 01 18:10:10.979647 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:10.979614 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" event={"ID":"bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e","Type":"ContainerStarted","Data":"f591276444b4ece7fad1cb8eea8389a4101470126baa39c14152382e48090d93"} Jul 01 18:10:11.279461 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:11.279375 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:11.279461 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:11.279427 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:11.279680 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.279518 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:19.27950087 +0000 UTC m=+228.420684220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : configmap references non-existent config key: service-ca.crt Jul 01 18:10:11.279680 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.279527 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 18:10:11.279680 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:11.279569 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:11.279680 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.279579 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:19.279563576 +0000 UTC m=+228.420746934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : secret "router-metrics-certs-default" not found Jul 01 18:10:11.279680 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.279643 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:11.279885 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.279698 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls podName:933d818a-0783-4070-9394-957b53f50751 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:19.279687133 +0000 UTC m=+228.420870484 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t7phw" (UID: "933d818a-0783-4070-9394-957b53f50751") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:11.380732 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:11.380699 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:11.380907 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.380877 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 18:10:11.380948 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.380933 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls podName:dfe2d982-946e-45e1-9cac-f3072e7c8917 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:19.380919551 +0000 UTC m=+228.522102903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-29swh" (UID: "dfe2d982-946e-45e1-9cac-f3072e7c8917") : secret "samples-operator-tls" not found Jul 01 18:10:11.582122 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:11.582040 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:11.582122 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:11.582106 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:11.582348 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.582176 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:10:11.582348 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.582245 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:10:11.582348 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.582256 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:13.582235167 +0000 UTC m=+222.723418517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : configmap "kube-rbac-proxy" not found Jul 01 18:10:11.582348 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:11.582294 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:13.582279469 +0000 UTC m=+222.723462819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : secret "insights-runtime-extractor-tls" not found Jul 01 18:10:11.631154 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:11.631128 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f5fv7_8528571c-8df9-4351-b771-3d37effe01ca/dns-node-resolver/0.log" Jul 01 18:10:12.988281 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:12.988244 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" event={"ID":"bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e","Type":"ContainerStarted","Data":"4f472a47d64eb3861c44ac95dfa91322e8d6b3c3d41d75de7e127b4395a70d3f"} Jul 01 18:10:12.988635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:12.988303 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" event={"ID":"bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e","Type":"ContainerStarted","Data":"eff48347eea31fefdb16299e5c0c4e36c2db7ae68099747d5212c95ced462cb7"} Jul 01 18:10:13.005338 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:13.005291 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-hc7x6" podStartSLOduration=1.63495586 podStartE2EDuration="3.005276734s" totalImagesPullingTime="1.370320874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:13.004198249 +0000 UTC m=+222.145381620" watchObservedRunningTime="2026-07-01 18:10:13.005276734 +0000 UTC m=+222.146460140" Jul 01 18:10:13.030469 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:13.030447 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lnrs5_9de04a82-ce0c-4dfa-b8e1-1db529bdfbab/node-ca/0.log" Jul 01 18:10:13.598608 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:13.598576 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:13.598793 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:13.598627 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:13.598793 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:13.598696 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:10:13.598793 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:13.598744 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:10:13.598793 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:13.598758 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:17.598742579 +0000 UTC m=+226.739925929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : configmap "kube-rbac-proxy" not found Jul 01 18:10:13.598955 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:13.598806 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:17.598790795 +0000 UTC m=+226.739974145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : secret "insights-runtime-extractor-tls" not found Jul 01 18:10:13.831832 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:13.831798 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zbkc9_c6c1a534-badc-41f6-95a8-2ebcba6394fc/serve-healthcheck-canary/0.log" Jul 01 18:10:17.628622 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:17.628571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:17.629039 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:17.628678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:17.629039 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:17.628736 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:10:17.629039 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:17.628753 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:10:17.629039 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:17.628825 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:25.628805637 +0000 UTC m=+234.769988989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : secret "insights-runtime-extractor-tls" not found Jul 01 18:10:17.629039 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:17.628844 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm podName:44927caa-061c-4efe-a8e3-4ff490799539 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:25.62883792 +0000 UTC m=+234.770021270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwwtp" (UID: "44927caa-061c-4efe-a8e3-4ff490799539") : configmap "kube-rbac-proxy" not found Jul 01 18:10:19.342316 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:19.342281 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:19.342689 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:19.342338 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:19.342689 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:19.342393 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:19.342689 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:19.342438 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 18:10:19.342689 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:19.342500 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:35.342482698 +0000 UTC m=+244.483666058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : secret "router-metrics-certs-default" not found Jul 01 18:10:19.342689 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:19.342517 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle podName:3c23be7d-3674-4403-b8ba-31854ca8de3c nodeName:}" failed. No retries permitted until 2026-07-01 18:10:35.342508155 +0000 UTC m=+244.483691509 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle") pod "router-default-86dc5bb4b9-fkb96" (UID: "3c23be7d-3674-4403-b8ba-31854ca8de3c") : configmap references non-existent config key: service-ca.crt Jul 01 18:10:19.342689 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:19.342523 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:19.342689 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:19.342556 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls podName:933d818a-0783-4070-9394-957b53f50751 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:35.342547796 +0000 UTC m=+244.483731145 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-t7phw" (UID: "933d818a-0783-4070-9394-957b53f50751") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:10:19.443047 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:19.443016 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:19.445331 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:19.445303 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfe2d982-946e-45e1-9cac-f3072e7c8917-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-29swh\" (UID: \"dfe2d982-946e-45e1-9cac-f3072e7c8917\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:19.510588 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:19.510566 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" Jul 01 18:10:19.634074 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:19.633934 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh"] Jul 01 18:10:20.010420 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:20.010336 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" event={"ID":"dfe2d982-946e-45e1-9cac-f3072e7c8917","Type":"ContainerStarted","Data":"4448ccf869df0f71393da1768960f2073ac8e05323e5a7cd8d386c8bfdecf5e5"} Jul 01 18:10:23.022435 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:23.022399 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" event={"ID":"dfe2d982-946e-45e1-9cac-f3072e7c8917","Type":"ContainerStarted","Data":"0292a3afdfc1e700461e5b9c3ffa81c9daa8a6c4db7773f3622ae0a7d07d6296"} Jul 01 18:10:23.022883 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:23.022448 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" event={"ID":"dfe2d982-946e-45e1-9cac-f3072e7c8917","Type":"ContainerStarted","Data":"19140aaacd3e860c41d38be4c4d982aeff940a31667fa4a0d9a41aa1b19098b3"} Jul 01 18:10:23.038531 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:23.038484 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-29swh" podStartSLOduration=17.746255694 podStartE2EDuration="20.038469867s" totalImagesPullingTime="2.292214173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:23.037654004 +0000 UTC m=+232.178837377" watchObservedRunningTime="2026-07-01 18:10:23.038469867 +0000 UTC m=+232.179653258" Jul 01 18:10:25.694619 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:25.694575 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:25.695057 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:25.694692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:25.695291 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:25.695269 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44927caa-061c-4efe-a8e3-4ff490799539-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:25.696998 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:25.696970 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44927caa-061c-4efe-a8e3-4ff490799539-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwwtp\" (UID: \"44927caa-061c-4efe-a8e3-4ff490799539\") " pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:25.752174 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:25.752144 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mwwtp" Jul 01 18:10:25.878640 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:25.878599 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mwwtp"] Jul 01 18:10:25.882644 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:25.882619 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44927caa_061c_4efe_a8e3_4ff490799539.slice/crio-0881f7dc535c41c3c83030d074d27d6f0c4cf6f513b30314f0cb6b0977a3a5b0 WatchSource:0}: Error finding container 0881f7dc535c41c3c83030d074d27d6f0c4cf6f513b30314f0cb6b0977a3a5b0: Status 404 returned error can't find the container with id 0881f7dc535c41c3c83030d074d27d6f0c4cf6f513b30314f0cb6b0977a3a5b0 Jul 01 18:10:26.032545 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:26.032463 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwwtp" event={"ID":"44927caa-061c-4efe-a8e3-4ff490799539","Type":"ContainerStarted","Data":"0881f7dc535c41c3c83030d074d27d6f0c4cf6f513b30314f0cb6b0977a3a5b0"} Jul 01 18:10:30.046568 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:30.046484 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwwtp" event={"ID":"44927caa-061c-4efe-a8e3-4ff490799539","Type":"ContainerStarted","Data":"eb382fdcb66a495044fde81af5243f43be6f24488845057f40b1ba2c2ff50d58"} Jul 01 18:10:31.050743 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:31.050706 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwwtp" event={"ID":"44927caa-061c-4efe-a8e3-4ff490799539","Type":"ContainerStarted","Data":"f4ec4d14d59aaf7eb98e653bbea5dab294311168eabee32db3d433d9148e16b0"} Jul 01 18:10:34.060934 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:34.060904 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwwtp" event={"ID":"44927caa-061c-4efe-a8e3-4ff490799539","Type":"ContainerStarted","Data":"80fb88f96e8f1dbba094b4ccf48ba735887dc1786a3b1017e705166387a75a43"} Jul 01 18:10:34.078263 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:34.078222 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mwwtp" podStartSLOduration=17.48566176 podStartE2EDuration="25.078208725s" totalImagesPullingTime="7.592546965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:10:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:34.077085357 +0000 UTC m=+243.218268724" watchObservedRunningTime="2026-07-01 18:10:34.078208725 +0000 UTC m=+243.219392096" Jul 01 18:10:35.366339 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.366307 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:35.366843 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.366356 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:35.366843 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.366380 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:35.367082 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.367058 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c23be7d-3674-4403-b8ba-31854ca8de3c-service-ca-bundle\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:35.368618 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.368594 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/933d818a-0783-4070-9394-957b53f50751-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-t7phw\" (UID: \"933d818a-0783-4070-9394-957b53f50751\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:35.368726 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.368647 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c23be7d-3674-4403-b8ba-31854ca8de3c-metrics-certs\") pod \"router-default-86dc5bb4b9-fkb96\" (UID: \"3c23be7d-3674-4403-b8ba-31854ca8de3c\") " pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:35.618225 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.618143 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ctw79\"" Jul 01 18:10:35.623986 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.623963 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2d7br\"" Jul 01 18:10:35.625860 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.625845 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" Jul 01 18:10:35.632932 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.632910 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:35.753685 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.753657 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw"] Jul 01 18:10:35.756964 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:35.756941 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933d818a_0783_4070_9394_957b53f50751.slice/crio-72453907208ba5278a521c98758aa5eb747141e83b060d079e97cb2f9ee1de30 WatchSource:0}: Error finding container 72453907208ba5278a521c98758aa5eb747141e83b060d079e97cb2f9ee1de30: Status 404 returned error can't find the container with id 72453907208ba5278a521c98758aa5eb747141e83b060d079e97cb2f9ee1de30 Jul 01 18:10:35.769791 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:35.769751 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86dc5bb4b9-fkb96"] Jul 01 18:10:35.772692 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:35.772661 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c23be7d_3674_4403_b8ba_31854ca8de3c.slice/crio-9166910dd298c6b669d597964a16c9c806631842962b40c8cef76f6baa129ef5 WatchSource:0}: Error finding container 9166910dd298c6b669d597964a16c9c806631842962b40c8cef76f6baa129ef5: Status 404 returned error can't find the container with id 9166910dd298c6b669d597964a16c9c806631842962b40c8cef76f6baa129ef5 Jul 01 18:10:36.068333 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:36.068296 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" event={"ID":"3c23be7d-3674-4403-b8ba-31854ca8de3c","Type":"ContainerStarted","Data":"24cd007fbe916faf71119610a31af585aaf79cb93e589742b38cd3f307b0f086"} Jul 01 18:10:36.068511 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:36.068352 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" event={"ID":"3c23be7d-3674-4403-b8ba-31854ca8de3c","Type":"ContainerStarted","Data":"9166910dd298c6b669d597964a16c9c806631842962b40c8cef76f6baa129ef5"} Jul 01 18:10:36.069383 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:36.069360 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" event={"ID":"933d818a-0783-4070-9394-957b53f50751","Type":"ContainerStarted","Data":"72453907208ba5278a521c98758aa5eb747141e83b060d079e97cb2f9ee1de30"} Jul 01 18:10:36.088727 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:36.088685 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" podStartSLOduration=33.088672566 podStartE2EDuration="33.088672566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:36.08746885 +0000 UTC m=+245.228652278" watchObservedRunningTime="2026-07-01 18:10:36.088672566 +0000 UTC m=+245.229855937" Jul 01 18:10:36.633738 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:36.633705 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:36.636241 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:36.636220 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:37.073855 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:37.073744 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:37.075166 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:37.075140 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-86dc5bb4b9-fkb96" Jul 01 18:10:38.077676 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.077627 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" event={"ID":"933d818a-0783-4070-9394-957b53f50751","Type":"ContainerStarted","Data":"b121213b2d7f87209e864a769e2ad77c2d03b9bd1c1ce7e01492b3e34bd57fb2"} Jul 01 18:10:38.095116 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.095067 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-t7phw" podStartSLOduration=32.844771644 podStartE2EDuration="35.095051711s" totalImagesPullingTime="2.250280067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:38.093543976 +0000 UTC m=+247.234727348" watchObservedRunningTime="2026-07-01 18:10:38.095051711 +0000 UTC m=+247.236235061" Jul 01 18:10:38.492334 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.492301 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4"] Jul 01 18:10:38.495383 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.495367 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" Jul 01 18:10:38.497665 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.497643 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 01 18:10:38.497756 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.497648 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5dfjx\"" Jul 01 18:10:38.505452 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.505429 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4"] Jul 01 18:10:38.590073 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.590039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2d208d24-280f-4f55-98dd-b3e38415885c-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-xrgk4\" (UID: \"2d208d24-280f-4f55-98dd-b3e38415885c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" Jul 01 18:10:38.691345 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.691312 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2d208d24-280f-4f55-98dd-b3e38415885c-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-xrgk4\" (UID: \"2d208d24-280f-4f55-98dd-b3e38415885c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" Jul 01 18:10:38.693583 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.693567 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2d208d24-280f-4f55-98dd-b3e38415885c-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-xrgk4\" (UID: \"2d208d24-280f-4f55-98dd-b3e38415885c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" Jul 01 18:10:38.804372 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.804279 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" Jul 01 18:10:38.925572 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:38.925551 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4"] Jul 01 18:10:38.927826 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:38.927798 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d208d24_280f_4f55_98dd_b3e38415885c.slice/crio-af3d6f541e4292902ee3e165fba66d04e0774323cc461359d1b178b3a6ae6914 WatchSource:0}: Error finding container af3d6f541e4292902ee3e165fba66d04e0774323cc461359d1b178b3a6ae6914: Status 404 returned error can't find the container with id af3d6f541e4292902ee3e165fba66d04e0774323cc461359d1b178b3a6ae6914 Jul 01 18:10:39.081896 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:39.081800 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" event={"ID":"2d208d24-280f-4f55-98dd-b3e38415885c","Type":"ContainerStarted","Data":"af3d6f541e4292902ee3e165fba66d04e0774323cc461359d1b178b3a6ae6914"} Jul 01 18:10:41.091420 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.091388 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" event={"ID":"2d208d24-280f-4f55-98dd-b3e38415885c","Type":"ContainerStarted","Data":"4de1e9b9ebbc9c4db893e8926c62be4d20931d35d46b4987faa0ecf907461348"} Jul 01 18:10:41.091906 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.091612 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" Jul 01 18:10:41.096226 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.096205 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" Jul 01 18:10:41.151231 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.151181 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-xrgk4" podStartSLOduration=1.815741617 podStartE2EDuration="3.151165648s" totalImagesPullingTime="1.335424031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:41.128617407 +0000 UTC m=+250.269800779" watchObservedRunningTime="2026-07-01 18:10:41.151165648 +0000 UTC m=+250.292349020" Jul 01 18:10:41.559905 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.559874 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-kvkfc"] Jul 01 18:10:41.563046 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.563031 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.565566 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.565549 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 01 18:10:41.565675 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.565550 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nlhhc\"" Jul 01 18:10:41.566553 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.566539 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 18:10:41.566648 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.566574 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 01 18:10:41.571093 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.571072 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-kvkfc"] Jul 01 18:10:41.611531 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.611500 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.611686 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.611539 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.611686 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.611562 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47h72\" (UniqueName: \"kubernetes.io/projected/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-kube-api-access-47h72\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.611686 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.611649 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.712324 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.712295 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.712324 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.712329 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47h72\" (UniqueName: \"kubernetes.io/projected/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-kube-api-access-47h72\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.712507 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.712460 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.712582 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.712564 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.712623 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:41.712576 2572 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 01 18:10:41.712660 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:41.712641 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-tls podName:3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:42.212626337 +0000 UTC m=+251.353809691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-kvkfc" (UID: "3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7") : secret "prometheus-operator-tls" not found Jul 01 18:10:41.713197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.713182 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.714809 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.714791 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:41.722435 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:41.722415 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47h72\" (UniqueName: \"kubernetes.io/projected/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-kube-api-access-47h72\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:42.216271 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:42.216235 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:42.218641 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:42.218617 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-kvkfc\" (UID: \"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:42.472182 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:42.472089 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" Jul 01 18:10:42.593414 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:42.593393 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-kvkfc"] Jul 01 18:10:42.595865 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:42.595817 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0f7491_df74_4f2a_b6d3_fe3b6ffd77a7.slice/crio-29505033c93e09987991f0579818ac6cebc1190a4ee1af8888b76547f222236d WatchSource:0}: Error finding container 29505033c93e09987991f0579818ac6cebc1190a4ee1af8888b76547f222236d: Status 404 returned error can't find the container with id 29505033c93e09987991f0579818ac6cebc1190a4ee1af8888b76547f222236d Jul 01 18:10:43.099431 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:43.099397 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" event={"ID":"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7","Type":"ContainerStarted","Data":"29505033c93e09987991f0579818ac6cebc1190a4ee1af8888b76547f222236d"} Jul 01 18:10:43.122935 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:43.122912 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:10:43.125167 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:43.125139 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7e0bb01-e2e5-41b6-a552-ae7b6380eaec-metrics-certs\") pod \"network-metrics-daemon-dr9zg\" (UID: \"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec\") " pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:10:43.259214 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:43.259183 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bhlcl\"" Jul 01 18:10:43.267283 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:43.267258 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dr9zg" Jul 01 18:10:43.404632 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:43.404578 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dr9zg"] Jul 01 18:10:43.406944 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:43.406923 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e0bb01_e2e5_41b6_a552_ae7b6380eaec.slice/crio-ef763311de90f487b52a5e4639bd52d01dc5c77aff5242f2c2148191326024d1 WatchSource:0}: Error finding container ef763311de90f487b52a5e4639bd52d01dc5c77aff5242f2c2148191326024d1: Status 404 returned error can't find the container with id ef763311de90f487b52a5e4639bd52d01dc5c77aff5242f2c2148191326024d1 Jul 01 18:10:44.104519 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:44.104490 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dr9zg" event={"ID":"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec","Type":"ContainerStarted","Data":"ef763311de90f487b52a5e4639bd52d01dc5c77aff5242f2c2148191326024d1"} Jul 01 18:10:45.110763 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:45.110729 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" event={"ID":"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7","Type":"ContainerStarted","Data":"0c27f880ae93787d6ee55d63d8d8a8aafc232b3252dabaef371c80aa14c63201"} Jul 01 18:10:45.111166 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:45.110808 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" event={"ID":"3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7","Type":"ContainerStarted","Data":"f23ee04ec1bf9b907cfbbdbeaf950a79d43edfb39bd71d5bb24e6342f0f682da"} Jul 01 18:10:45.127192 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:45.127075 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-kvkfc" podStartSLOduration=2.117904909 podStartE2EDuration="4.127055857s" totalImagesPullingTime="2.009150948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:45.126565808 +0000 UTC m=+254.267749179" watchObservedRunningTime="2026-07-01 18:10:45.127055857 +0000 UTC m=+254.268239232" Jul 01 18:10:46.115111 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.115074 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dr9zg" event={"ID":"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec","Type":"ContainerStarted","Data":"dce77039f438e93e3995ae67222f9ddc9f2e284e960ca6ba6eeb736be73fcad2"} Jul 01 18:10:46.115472 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.115130 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dr9zg" event={"ID":"d7e0bb01-e2e5-41b6-a552-ae7b6380eaec","Type":"ContainerStarted","Data":"7b6c195eaeb46c5c68760522813672ad66006a2c8925eef30ee4157eb1dac728"} Jul 01 18:10:46.130658 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.130621 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dr9zg" podStartSLOduration=253.066329009 podStartE2EDuration="4m15.13060725s" totalImagesPullingTime="2.064278241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:46.129561164 +0000 UTC m=+255.270744532" watchObservedRunningTime="2026-07-01 18:10:46.13060725 +0000 UTC m=+255.271790616" Jul 01 18:10:46.897567 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.897530 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k"] Jul 01 18:10:46.902504 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.901846 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:46.904373 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.904347 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 01 18:10:46.904509 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.904454 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-sqh47\"" Jul 01 18:10:46.904509 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.904480 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 01 18:10:46.904716 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.904696 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-x77tx"] Jul 01 18:10:46.907811 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.907794 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:46.910159 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.910140 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 01 18:10:46.910269 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.910212 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 01 18:10:46.910269 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.910259 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ltzxm\"" Jul 01 18:10:46.911429 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.911408 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 01 18:10:46.913265 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.913246 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k"] Jul 01 18:10:46.922174 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.922156 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-x77tx"] Jul 01 18:10:46.929015 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.928995 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9qzrn"] Jul 01 18:10:46.932119 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.932104 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:46.934438 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.934420 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 18:10:46.934528 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.934464 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 18:10:46.934528 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.934486 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-js6g5\"" Jul 01 18:10:46.934800 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.934761 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 18:10:46.951337 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.951315 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad051676-d1e8-428f-853e-ee2b2082d4a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:46.951409 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.951346 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad051676-d1e8-428f-853e-ee2b2082d4a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:46.951409 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.951371 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5zbk\" (UniqueName: \"kubernetes.io/projected/ad051676-d1e8-428f-853e-ee2b2082d4a9-kube-api-access-b5zbk\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:46.951486 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:46.951465 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad051676-d1e8-428f-853e-ee2b2082d4a9-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.052032 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.051995 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5zbk\" (UniqueName: \"kubernetes.io/projected/ad051676-d1e8-428f-853e-ee2b2082d4a9-kube-api-access-b5zbk\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.052032 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052036 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97b391cd-4f83-4497-82de-54e288f77664-metrics-client-ca\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.052245 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052060 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-textfile\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052245 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052081 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-tls\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052245 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052135 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-wtmp\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052245 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052158 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbwvv\" (UniqueName: \"kubernetes.io/projected/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-kube-api-access-pbwvv\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052245 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052207 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/97b391cd-4f83-4497-82de-54e288f77664-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.052245 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052236 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.052493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052260 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-accelerators-collector-config\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052291 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-root\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052317 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad051676-d1e8-428f-853e-ee2b2082d4a9-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.052493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052330 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-sys\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052349 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052377 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad051676-d1e8-428f-853e-ee2b2082d4a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.052493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052462 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cmpd\" (UniqueName: \"kubernetes.io/projected/97b391cd-4f83-4497-82de-54e288f77664-kube-api-access-8cmpd\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.052493 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052484 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-metrics-client-ca\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.052868 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052511 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.052868 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052535 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad051676-d1e8-428f-853e-ee2b2082d4a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.052868 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.052569 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.053056 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.053036 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad051676-d1e8-428f-853e-ee2b2082d4a9-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.054900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.054875 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad051676-d1e8-428f-853e-ee2b2082d4a9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.054999 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.054885 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad051676-d1e8-428f-853e-ee2b2082d4a9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.060357 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.060321 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5zbk\" (UniqueName: \"kubernetes.io/projected/ad051676-d1e8-428f-853e-ee2b2082d4a9-kube-api-access-b5zbk\") pod \"openshift-state-metrics-57f99c8b49-rkv7k\" (UID: \"ad051676-d1e8-428f-853e-ee2b2082d4a9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.153135 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153067 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-wtmp\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153135 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153096 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbwvv\" (UniqueName: \"kubernetes.io/projected/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-kube-api-access-pbwvv\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153135 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153120 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/97b391cd-4f83-4497-82de-54e288f77664-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153142 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153171 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-accelerators-collector-config\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153241 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-wtmp\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:47.153274 2572 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:10:47.153344 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-tls podName:97b391cd-4f83-4497-82de-54e288f77664 nodeName:}" failed. No retries permitted until 2026-07-01 18:10:47.653323487 +0000 UTC m=+256.794506853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-tls") pod "kube-state-metrics-845b954455-x77tx" (UID: "97b391cd-4f83-4497-82de-54e288f77664") : secret "kube-state-metrics-tls" not found Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153380 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-root\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153411 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-sys\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153455 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153503 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/97b391cd-4f83-4497-82de-54e288f77664-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153517 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cmpd\" (UniqueName: \"kubernetes.io/projected/97b391cd-4f83-4497-82de-54e288f77664-kube-api-access-8cmpd\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153538 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-root\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-metrics-client-ca\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153606 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-sys\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.153635 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153625 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.154317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153674 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.154317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153715 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97b391cd-4f83-4497-82de-54e288f77664-metrics-client-ca\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.154317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153750 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-textfile\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.154317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153796 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-accelerators-collector-config\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.154317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.153802 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-tls\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.154317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.154090 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-metrics-client-ca\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.154317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.154144 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-textfile\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.154670 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.154411 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97b391cd-4f83-4497-82de-54e288f77664-metrics-client-ca\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.154670 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.154426 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.155869 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.155844 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.156094 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.156075 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-node-exporter-tls\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.156207 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.156187 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.161708 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.161663 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbwvv\" (UniqueName: \"kubernetes.io/projected/1f17c45b-6fb0-4003-9054-3fb5c3dc056e-kube-api-access-pbwvv\") pod \"node-exporter-9qzrn\" (UID: \"1f17c45b-6fb0-4003-9054-3fb5c3dc056e\") " pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.161708 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.161702 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cmpd\" (UniqueName: \"kubernetes.io/projected/97b391cd-4f83-4497-82de-54e288f77664-kube-api-access-8cmpd\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.213558 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.213528 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" Jul 01 18:10:47.239689 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.239663 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9qzrn" Jul 01 18:10:47.250956 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:47.250923 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f17c45b_6fb0_4003_9054_3fb5c3dc056e.slice/crio-bfcc0a2c5f6e3ab1ac02298b00f03bb10746d5dbaa4a9cdf20fc9235b4960496 WatchSource:0}: Error finding container bfcc0a2c5f6e3ab1ac02298b00f03bb10746d5dbaa4a9cdf20fc9235b4960496: Status 404 returned error can't find the container with id bfcc0a2c5f6e3ab1ac02298b00f03bb10746d5dbaa4a9cdf20fc9235b4960496 Jul 01 18:10:47.339040 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.339007 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k"] Jul 01 18:10:47.341707 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:47.341675 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad051676_d1e8_428f_853e_ee2b2082d4a9.slice/crio-b147e6b89b5174bb2aca34817f16ec6ea43d29645597d2ddbc8a813701e0c584 WatchSource:0}: Error finding container b147e6b89b5174bb2aca34817f16ec6ea43d29645597d2ddbc8a813701e0c584: Status 404 returned error can't find the container with id b147e6b89b5174bb2aca34817f16ec6ea43d29645597d2ddbc8a813701e0c584 Jul 01 18:10:47.657848 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.657737 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.660102 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.660074 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b391cd-4f83-4497-82de-54e288f77664-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-x77tx\" (UID: \"97b391cd-4f83-4497-82de-54e288f77664\") " pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.818917 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.818884 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" Jul 01 18:10:47.979918 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:47.979833 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-x77tx"] Jul 01 18:10:48.126150 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:48.126005 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" event={"ID":"ad051676-d1e8-428f-853e-ee2b2082d4a9","Type":"ContainerStarted","Data":"b1cc0fca3989992fa7ab3412751c7469c806936dfc172686b8ea338f5e02593c"} Jul 01 18:10:48.126150 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:48.126080 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" event={"ID":"ad051676-d1e8-428f-853e-ee2b2082d4a9","Type":"ContainerStarted","Data":"9475433e5d74fb7363e9b9cb1dd865c88635e5f0ede87535354434ee800878a7"} Jul 01 18:10:48.126150 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:48.126113 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" event={"ID":"ad051676-d1e8-428f-853e-ee2b2082d4a9","Type":"ContainerStarted","Data":"b147e6b89b5174bb2aca34817f16ec6ea43d29645597d2ddbc8a813701e0c584"} Jul 01 18:10:48.127737 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:48.127706 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9qzrn" event={"ID":"1f17c45b-6fb0-4003-9054-3fb5c3dc056e","Type":"ContainerStarted","Data":"bfcc0a2c5f6e3ab1ac02298b00f03bb10746d5dbaa4a9cdf20fc9235b4960496"} Jul 01 18:10:48.215142 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:48.213950 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b391cd_4f83_4497_82de_54e288f77664.slice/crio-f1612310a89f47ab72ce8b545163f83a82c40d9a53617dda98a1137c62b236a6 WatchSource:0}: Error finding container f1612310a89f47ab72ce8b545163f83a82c40d9a53617dda98a1137c62b236a6: Status 404 returned error can't find the container with id f1612310a89f47ab72ce8b545163f83a82c40d9a53617dda98a1137c62b236a6 Jul 01 18:10:49.134301 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.134261 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" event={"ID":"ad051676-d1e8-428f-853e-ee2b2082d4a9","Type":"ContainerStarted","Data":"d372f2748f7c52b8f96d583896c040e759e9716fc305de2982903d2ea6ce270d"} Jul 01 18:10:49.135785 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.135752 2572 generic.go:356] "Generic (PLEG): container finished" podID="1f17c45b-6fb0-4003-9054-3fb5c3dc056e" containerID="a955b9e59bb17f6328564520cd477e9379995357e66d13d07b3482ada87ec7f2" exitCode=0 Jul 01 18:10:49.135933 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.135832 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9qzrn" event={"ID":"1f17c45b-6fb0-4003-9054-3fb5c3dc056e","Type":"ContainerDied","Data":"a955b9e59bb17f6328564520cd477e9379995357e66d13d07b3482ada87ec7f2"} Jul 01 18:10:49.136982 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.136956 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" event={"ID":"97b391cd-4f83-4497-82de-54e288f77664","Type":"ContainerStarted","Data":"f1612310a89f47ab72ce8b545163f83a82c40d9a53617dda98a1137c62b236a6"} Jul 01 18:10:49.153849 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.153796 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-rkv7k" podStartSLOduration=2.102450991 podStartE2EDuration="3.153759685s" totalImagesPullingTime="1.051308694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:49.152914054 +0000 UTC m=+258.294097428" watchObservedRunningTime="2026-07-01 18:10:49.153759685 +0000 UTC m=+258.294943059" Jul 01 18:10:49.895075 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.895043 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-969f7c748-r26br"] Jul 01 18:10:49.898303 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.898288 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:49.900987 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.900966 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 01 18:10:49.900987 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.900985 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9erhj7h5ob4m2\"" Jul 01 18:10:49.901319 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.900971 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 01 18:10:49.901319 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.901020 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 01 18:10:49.901319 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.901058 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-54nrv\"" Jul 01 18:10:49.901319 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.901257 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 01 18:10:49.901319 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.901291 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 01 18:10:49.912739 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.912720 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-969f7c748-r26br"] Jul 01 18:10:49.975794 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.975383 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:49.975794 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.975451 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd8ln\" (UniqueName: \"kubernetes.io/projected/3eb0d90d-d86a-4182-941f-0986044e6265-kube-api-access-kd8ln\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:49.975794 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.975481 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:49.975794 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.975517 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb0d90d-d86a-4182-941f-0986044e6265-metrics-client-ca\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:49.975794 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.975540 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-grpc-tls\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:49.975794 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.975577 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:49.975794 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.975609 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:49.975794 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:49.975639 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-tls\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.076333 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.076304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd8ln\" (UniqueName: \"kubernetes.io/projected/3eb0d90d-d86a-4182-941f-0986044e6265-kube-api-access-kd8ln\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.076487 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.076349 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.076487 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.076390 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb0d90d-d86a-4182-941f-0986044e6265-metrics-client-ca\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.076487 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.076413 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-grpc-tls\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.076487 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.076448 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.076487 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.076480 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.076713 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.076508 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-tls\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.076713 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.076585 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.078045 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.078013 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb0d90d-d86a-4182-941f-0986044e6265-metrics-client-ca\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.080132 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.080089 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-tls\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.080760 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.080690 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.080880 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.080756 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.080880 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.080816 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.080880 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.080803 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.081324 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.081307 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eb0d90d-d86a-4182-941f-0986044e6265-secret-grpc-tls\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.085078 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.085046 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd8ln\" (UniqueName: \"kubernetes.io/projected/3eb0d90d-d86a-4182-941f-0986044e6265-kube-api-access-kd8ln\") pod \"thanos-querier-969f7c748-r26br\" (UID: \"3eb0d90d-d86a-4182-941f-0986044e6265\") " pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.143437 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.143398 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9qzrn" event={"ID":"1f17c45b-6fb0-4003-9054-3fb5c3dc056e","Type":"ContainerStarted","Data":"c922f3eeee7c1147253e24c16e0cf64e18edaa32ed55a6cd277fd25e0c737caf"} Jul 01 18:10:50.143607 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.143468 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9qzrn" event={"ID":"1f17c45b-6fb0-4003-9054-3fb5c3dc056e","Type":"ContainerStarted","Data":"9437a8fe18e5cd8808f382b83f4e020d01abff6c5819637b6e5fcd91660ac9cd"} Jul 01 18:10:50.146874 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.146764 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" event={"ID":"97b391cd-4f83-4497-82de-54e288f77664","Type":"ContainerStarted","Data":"75f17bb33b5f23d476cb1575b1857dfe9cfe3cc8b6ed895dda57f7d460691099"} Jul 01 18:10:50.146874 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.146853 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" event={"ID":"97b391cd-4f83-4497-82de-54e288f77664","Type":"ContainerStarted","Data":"7d489b4635c0b2a7f72c2feea5646c16dd2ab6b8151315f34dd997eca7e59d0b"} Jul 01 18:10:50.147056 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.146881 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" event={"ID":"97b391cd-4f83-4497-82de-54e288f77664","Type":"ContainerStarted","Data":"bc5db0cfd55a8f2effac6e89a687a573b162fa38ffdb714d443550821b22821f"} Jul 01 18:10:50.163600 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.163548 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9qzrn" podStartSLOduration=2.886575847 podStartE2EDuration="4.163526259s" totalImagesPullingTime="1.210990543s" totalInitContainerRuntime="65.959869ms" isStatefulPod=true podCreationTimestamp="2026-07-01 18:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:50.161962089 +0000 UTC m=+259.303145473" watchObservedRunningTime="2026-07-01 18:10:50.163526259 +0000 UTC m=+259.304709632" Jul 01 18:10:50.179256 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.179214 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-x77tx" podStartSLOduration=2.4893625999999998 podStartE2EDuration="4.179194655s" totalImagesPullingTime="1.689832055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:50.178539659 +0000 UTC m=+259.319723032" watchObservedRunningTime="2026-07-01 18:10:50.179194655 +0000 UTC m=+259.320378029" Jul 01 18:10:50.207958 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.207916 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:50.343381 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:50.343298 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-969f7c748-r26br"] Jul 01 18:10:50.345658 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:50.345621 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb0d90d_d86a_4182_941f_0986044e6265.slice/crio-5f8e8d3a25313c9ae04a96abdfedcd0cbd4fc57405b5f68ea2cfd23015e0a1f6 WatchSource:0}: Error finding container 5f8e8d3a25313c9ae04a96abdfedcd0cbd4fc57405b5f68ea2cfd23015e0a1f6: Status 404 returned error can't find the container with id 5f8e8d3a25313c9ae04a96abdfedcd0cbd4fc57405b5f68ea2cfd23015e0a1f6 Jul 01 18:10:51.151757 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:51.151711 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" event={"ID":"3eb0d90d-d86a-4182-941f-0986044e6265","Type":"ContainerStarted","Data":"5f8e8d3a25313c9ae04a96abdfedcd0cbd4fc57405b5f68ea2cfd23015e0a1f6"} Jul 01 18:10:53.131674 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.131598 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:10:53.135602 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.135579 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.138148 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.138125 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 18:10:53.138263 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.138198 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6l3kpjv945fnf\"" Jul 01 18:10:53.138554 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.138503 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 18:10:53.138645 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.138617 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 18:10:53.138877 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.138857 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 18:10:53.138977 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.138915 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 18:10:53.139389 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.139329 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 18:10:53.139487 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.139395 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 18:10:53.139487 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.139410 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 18:10:53.139594 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.139532 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 18:10:53.139594 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.139547 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 18:10:53.139594 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.139568 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 18:10:53.139750 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.139574 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 18:10:53.139750 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.139663 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 18:10:53.140334 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.140195 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m7fzl\"" Jul 01 18:10:53.142718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.142698 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 18:10:53.146439 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.146421 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 18:10:53.151226 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.151206 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:10:53.202233 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202206 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202233 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202242 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202467 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202304 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202467 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202347 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202467 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202382 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202467 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202408 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-web-config\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202467 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202431 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202467 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202457 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202481 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202515 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202547 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-config-out\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202588 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202626 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202645 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202675 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202719 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-config\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.202749 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202747 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.203066 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202761 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-662mj\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-kube-api-access-662mj\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.203066 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202802 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.203066 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.202824 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.303857 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.303819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304057 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.303881 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304057 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.303915 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304057 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.303961 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304057 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.303990 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304057 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-web-config\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304057 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304050 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304092 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304119 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304144 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304197 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-config-out\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304260 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304286 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304314 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304374 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-config\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304402 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-662mj\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-kube-api-access-662mj\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304452 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.304705 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.304489 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.305270 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.305003 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308472 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.306359 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308472 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.306603 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308472 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.307010 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308472 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.307723 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308472 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.308068 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308472 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.308399 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308472 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.308432 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308921 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.308506 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.308921 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.308845 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.309322 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.309296 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.309745 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.309706 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-config\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.310372 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.310338 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.310452 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.310378 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-web-config\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.311069 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.311047 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.311164 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.311071 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-config-out\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.311270 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.311244 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.311389 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.311368 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.311683 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.311662 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.317910 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.317887 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-662mj\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-kube-api-access-662mj\") pod \"prometheus-k8s-0\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.447765 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.447670 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:10:53.606567 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:53.606537 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:10:53.609474 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:10:53.609440 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaea7031_37dc_491f_97d1_c9efc7bd20f7.slice/crio-19edd48bb9ab5217d5e21a6202d63696ee7c639bbdf15bcf741f71c9739ec8d4 WatchSource:0}: Error finding container 19edd48bb9ab5217d5e21a6202d63696ee7c639bbdf15bcf741f71c9739ec8d4: Status 404 returned error can't find the container with id 19edd48bb9ab5217d5e21a6202d63696ee7c639bbdf15bcf741f71c9739ec8d4 Jul 01 18:10:54.165917 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:54.165874 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerStarted","Data":"19edd48bb9ab5217d5e21a6202d63696ee7c639bbdf15bcf741f71c9739ec8d4"} Jul 01 18:10:55.171739 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:55.171707 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" event={"ID":"3eb0d90d-d86a-4182-941f-0986044e6265","Type":"ContainerStarted","Data":"646f0934799ed891ed2cb6972143626538904872b6eb03c7dddbe8f7d65366af"} Jul 01 18:10:55.172108 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:55.171787 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" event={"ID":"3eb0d90d-d86a-4182-941f-0986044e6265","Type":"ContainerStarted","Data":"7ee4469b2bfaa52c265bf87a993dc08ff3b3d1bba529bdcb9abe19bd452828dd"} Jul 01 18:10:55.172108 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:55.171815 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" event={"ID":"3eb0d90d-d86a-4182-941f-0986044e6265","Type":"ContainerStarted","Data":"874d3542d9958ea80a204733797032f09f910d38a39a7451357b0623b7ccae1c"} Jul 01 18:10:56.177536 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:56.177501 2572 generic.go:356] "Generic (PLEG): container finished" podID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" exitCode=0 Jul 01 18:10:56.178001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:56.177591 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerDied","Data":"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982"} Jul 01 18:10:57.184457 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:57.184423 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" event={"ID":"3eb0d90d-d86a-4182-941f-0986044e6265","Type":"ContainerStarted","Data":"731ef696e3ac21da152307c4107611c20e9dacc5803b2a712888e747001ae45d"} Jul 01 18:10:57.184914 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:57.184485 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" event={"ID":"3eb0d90d-d86a-4182-941f-0986044e6265","Type":"ContainerStarted","Data":"17a3a3a29c1243b29aab8aa77a4d96851ac01bd58914ac562bd2b1ca9254a584"} Jul 01 18:10:57.184914 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:57.184511 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" event={"ID":"3eb0d90d-d86a-4182-941f-0986044e6265","Type":"ContainerStarted","Data":"7be4f59048c692031fc861c9c40b7bcbb51e8f77deb23619d79ec016184a5e32"} Jul 01 18:10:57.184914 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:57.184570 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:10:57.206010 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:10:57.205956 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" podStartSLOduration=2.536216214 podStartE2EDuration="8.20593866s" totalImagesPullingTime="5.669722446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:10:57.204307335 +0000 UTC m=+266.345490700" watchObservedRunningTime="2026-07-01 18:10:57.20593866 +0000 UTC m=+266.347122033" Jul 01 18:11:01.202735 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:01.202696 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerStarted","Data":"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726"} Jul 01 18:11:01.203090 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:01.202795 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerStarted","Data":"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26"} Jul 01 18:11:01.203090 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:01.202837 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerStarted","Data":"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c"} Jul 01 18:11:01.203090 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:01.202880 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerStarted","Data":"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797"} Jul 01 18:11:01.203090 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:01.202915 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerStarted","Data":"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27"} Jul 01 18:11:01.203090 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:01.202952 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerStarted","Data":"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d"} Jul 01 18:11:01.232107 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:01.232054 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.358580461 podStartE2EDuration="8.232037549s" totalImagesPullingTime="5.854864007s" totalInitContainerRuntime="18.593081ms" isStatefulPod=false podCreationTimestamp="2026-07-01 18:10:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:11:01.229504236 +0000 UTC m=+270.370687607" watchObservedRunningTime="2026-07-01 18:11:01.232037549 +0000 UTC m=+270.373220921" Jul 01 18:11:03.194204 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:03.194176 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-969f7c748-r26br" Jul 01 18:11:03.448693 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:03.448611 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:11:07.825576 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:11:07.825528 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wmv7g" podUID="2c83a996-6382-4768-ba63-7d3a5067953a" Jul 01 18:11:08.230274 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:08.230245 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wmv7g" Jul 01 18:11:11.459813 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:11.459765 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:11:11.462139 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:11.462117 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c83a996-6382-4768-ba63-7d3a5067953a-metrics-tls\") pod \"dns-default-wmv7g\" (UID: \"2c83a996-6382-4768-ba63-7d3a5067953a\") " pod="openshift-dns/dns-default-wmv7g" Jul 01 18:11:11.533699 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:11.533676 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vsbfc\"" Jul 01 18:11:11.541648 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:11.541629 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wmv7g" Jul 01 18:11:11.680408 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:11.680284 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wmv7g"] Jul 01 18:11:11.683151 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:11:11.683121 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c83a996_6382_4768_ba63_7d3a5067953a.slice/crio-09eedc6622748e4cab154f0c9321d95ba5a871e0daec87d107b89344c284c901 WatchSource:0}: Error finding container 09eedc6622748e4cab154f0c9321d95ba5a871e0daec87d107b89344c284c901: Status 404 returned error can't find the container with id 09eedc6622748e4cab154f0c9321d95ba5a871e0daec87d107b89344c284c901 Jul 01 18:11:12.246788 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:12.246743 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wmv7g" event={"ID":"2c83a996-6382-4768-ba63-7d3a5067953a","Type":"ContainerStarted","Data":"09eedc6622748e4cab154f0c9321d95ba5a871e0daec87d107b89344c284c901"} Jul 01 18:11:15.260203 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:15.260164 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wmv7g" event={"ID":"2c83a996-6382-4768-ba63-7d3a5067953a","Type":"ContainerStarted","Data":"b20b82df6c2f3c83dc33f97f0b6684ef9ee0329ee468521b76564d68b918eae6"} Jul 01 18:11:15.260625 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:15.260262 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wmv7g" event={"ID":"2c83a996-6382-4768-ba63-7d3a5067953a","Type":"ContainerStarted","Data":"8a4f46fe4ce215445d69fc2c9e40f0a6489a73f49f73ed7c982f36ed004ee667"} Jul 01 18:11:15.260625 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:15.260311 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wmv7g" Jul 01 18:11:15.277384 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:15.277336 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wmv7g" podStartSLOduration=251.723487597 podStartE2EDuration="4m14.277321666s" totalImagesPullingTime="2.553834069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:11:15.27611909 +0000 UTC m=+284.417302464" watchObservedRunningTime="2026-07-01 18:11:15.277321666 +0000 UTC m=+284.418505072" Jul 01 18:11:23.294670 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:23.294634 2572 generic.go:356] "Generic (PLEG): container finished" podID="2af814a3-00a8-4ace-97b7-5c069a6cede2" containerID="cc9685dde56c0766139d4919b65cb29aa377fd5426d6bc1472f7b26b4c677dfa" exitCode=0 Jul 01 18:11:23.295071 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:23.294695 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" event={"ID":"2af814a3-00a8-4ace-97b7-5c069a6cede2","Type":"ContainerDied","Data":"cc9685dde56c0766139d4919b65cb29aa377fd5426d6bc1472f7b26b4c677dfa"} Jul 01 18:11:23.295112 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:23.295092 2572 scope.go:117] "RemoveContainer" containerID="cc9685dde56c0766139d4919b65cb29aa377fd5426d6bc1472f7b26b4c677dfa" Jul 01 18:11:23.827407 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:23.827371 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-t7phw_933d818a-0783-4070-9394-957b53f50751/cluster-monitoring-operator/0.log" Jul 01 18:11:24.026057 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:24.026024 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-x77tx_97b391cd-4f83-4497-82de-54e288f77664/kube-state-metrics/0.log" Jul 01 18:11:24.227539 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:24.227503 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-x77tx_97b391cd-4f83-4497-82de-54e288f77664/kube-rbac-proxy-main/0.log" Jul 01 18:11:24.301078 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:24.301043 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-qgjgz" event={"ID":"2af814a3-00a8-4ace-97b7-5c069a6cede2","Type":"ContainerStarted","Data":"b15362e4f5f60055da9b2c91fb3ac8c8a76f5634e6fa819ba3f4b3c56478e327"} Jul 01 18:11:24.425974 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:24.425949 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-x77tx_97b391cd-4f83-4497-82de-54e288f77664/kube-rbac-proxy-self/0.log" Jul 01 18:11:25.266627 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:25.266590 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wmv7g" Jul 01 18:11:25.625879 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:25.625858 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qzrn_1f17c45b-6fb0-4003-9054-3fb5c3dc056e/init-textfile/0.log" Jul 01 18:11:25.826366 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:25.826337 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qzrn_1f17c45b-6fb0-4003-9054-3fb5c3dc056e/node-exporter/0.log" Jul 01 18:11:26.025679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:26.025602 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qzrn_1f17c45b-6fb0-4003-9054-3fb5c3dc056e/kube-rbac-proxy/0.log" Jul 01 18:11:26.825481 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:26.825449 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-rkv7k_ad051676-d1e8-428f-853e-ee2b2082d4a9/kube-rbac-proxy-main/0.log" Jul 01 18:11:27.025953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:27.025924 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-rkv7k_ad051676-d1e8-428f-853e-ee2b2082d4a9/kube-rbac-proxy-self/0.log" Jul 01 18:11:27.225856 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:27.225831 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-rkv7k_ad051676-d1e8-428f-853e-ee2b2082d4a9/openshift-state-metrics/0.log" Jul 01 18:11:27.426549 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:27.426520 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_baea7031-37dc-491f-97d1-c9efc7bd20f7/init-config-reloader/0.log" Jul 01 18:11:27.626413 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:27.626388 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_baea7031-37dc-491f-97d1-c9efc7bd20f7/prometheus/0.log" Jul 01 18:11:27.825850 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:27.825823 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_baea7031-37dc-491f-97d1-c9efc7bd20f7/config-reloader/0.log" Jul 01 18:11:28.026055 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:28.025984 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_baea7031-37dc-491f-97d1-c9efc7bd20f7/thanos-sidecar/0.log" Jul 01 18:11:28.226855 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:28.226821 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_baea7031-37dc-491f-97d1-c9efc7bd20f7/kube-rbac-proxy-web/0.log" Jul 01 18:11:28.426793 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:28.426748 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_baea7031-37dc-491f-97d1-c9efc7bd20f7/kube-rbac-proxy/0.log" Jul 01 18:11:28.627179 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:28.627146 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_baea7031-37dc-491f-97d1-c9efc7bd20f7/kube-rbac-proxy-thanos/0.log" Jul 01 18:11:28.827280 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:28.827211 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-kvkfc_3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7/prometheus-operator/0.log" Jul 01 18:11:29.027485 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:29.027456 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-kvkfc_3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7/kube-rbac-proxy/0.log" Jul 01 18:11:29.225066 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:29.225041 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-xrgk4_2d208d24-280f-4f55-98dd-b3e38415885c/prometheus-operator-admission-webhook/0.log" Jul 01 18:11:29.425540 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:29.425513 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/thanos-query/0.log" Jul 01 18:11:29.625282 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:29.625256 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/kube-rbac-proxy-web/0.log" Jul 01 18:11:29.825580 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:29.825541 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/kube-rbac-proxy/0.log" Jul 01 18:11:30.026050 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:30.025978 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/prom-label-proxy/0.log" Jul 01 18:11:30.226287 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:30.226262 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/kube-rbac-proxy-rules/0.log" Jul 01 18:11:30.425327 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:30.425296 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/kube-rbac-proxy-metrics/0.log" Jul 01 18:11:31.310656 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:31.310630 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 18:11:32.625562 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:32.625532 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wmv7g_2c83a996-6382-4768-ba63-7d3a5067953a/dns/0.log" Jul 01 18:11:32.826088 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:32.826062 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wmv7g_2c83a996-6382-4768-ba63-7d3a5067953a/kube-rbac-proxy/0.log" Jul 01 18:11:33.025630 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:33.025565 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f5fv7_8528571c-8df9-4351-b771-3d37effe01ca/dns-node-resolver/0.log" Jul 01 18:11:34.424921 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:34.424893 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lnrs5_9de04a82-ce0c-4dfa-b8e1-1db529bdfbab/node-ca/0.log" Jul 01 18:11:34.625762 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:34.625734 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86dc5bb4b9-fkb96_3c23be7d-3674-4403-b8ba-31854ca8de3c/router/0.log" Jul 01 18:11:35.225656 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:35.225630 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zbkc9_c6c1a534-badc-41f6-95a8-2ebcba6394fc/serve-healthcheck-canary/0.log" Jul 01 18:11:53.448793 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:53.448729 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:11:53.464084 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:53.464059 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:11:54.439589 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:11:54.439562 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:11.503139 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.503110 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:12:11.503679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.503403 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="prometheus" containerID="cri-o://6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" gracePeriod=600 Jul 01 18:12:11.503679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.503449 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="thanos-sidecar" containerID="cri-o://6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" gracePeriod=600 Jul 01 18:12:11.503679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.503447 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy" containerID="cri-o://bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" gracePeriod=600 Jul 01 18:12:11.503679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.503505 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="config-reloader" containerID="cri-o://d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" gracePeriod=600 Jul 01 18:12:11.503679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.503542 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" gracePeriod=600 Jul 01 18:12:11.503679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.503463 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy-web" containerID="cri-o://d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" gracePeriod=600 Jul 01 18:12:11.764737 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.764713 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:11.857361 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857334 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-2\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857361 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857365 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-kubelet-serving-ca-bundle\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857383 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-662mj\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-kube-api-access-662mj\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857404 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-kube-rbac-proxy\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857429 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-grpc-tls\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857468 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-config-out\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857491 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-1\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857528 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-tls\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857556 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857577 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-config\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857605 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-trusted-ca-bundle\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857656 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-metrics-client-certs\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857689 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-metrics-client-ca\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857725 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857753 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-serving-certs-ca-bundle\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857818 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857832 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-tls-assets\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.857900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857904 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-web-config\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.858718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857962 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-0\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.858718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.857988 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-thanos-prometheus-http-client-file\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.858718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.858030 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-db\") pod \"baea7031-37dc-491f-97d1-c9efc7bd20f7\" (UID: \"baea7031-37dc-491f-97d1-c9efc7bd20f7\") " Jul 01 18:12:11.858718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.858092 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:12:11.858718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.858300 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.858718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.858321 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.858718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.858335 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:12:11.858718 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.858664 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:12:11.859132 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.858965 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:12:11.859132 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.859118 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:12:11.860131 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.860099 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:12:11.860409 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.860382 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:12:11.860907 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.860882 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:12:11.861021 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.860986 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.861102 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.861087 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-config" (OuterVolumeSpecName: "config") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.861167 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.861146 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.861676 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.861643 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.861879 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.861846 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-kube-api-access-662mj" (OuterVolumeSpecName: "kube-api-access-662mj") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "kube-api-access-662mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:12:11.861962 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.861944 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-config-out" (OuterVolumeSpecName: "config-out") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:12:11.862402 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.862380 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.862620 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.862599 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.862928 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.862908 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.863119 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.863091 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.872746 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.872723 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-web-config" (OuterVolumeSpecName: "web-config") pod "baea7031-37dc-491f-97d1-c9efc7bd20f7" (UID: "baea7031-37dc-491f-97d1-c9efc7bd20f7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:12:11.959639 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959614 2572 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-config-out\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959639 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959636 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959646 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959656 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959665 2572 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-config\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959673 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959681 2572 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-metrics-client-certs\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959691 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-metrics-client-ca\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959699 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959709 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959717 2572 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-tls-assets\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959725 2572 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-web-config\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959732 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959740 2572 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959748 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/baea7031-37dc-491f-97d1-c9efc7bd20f7-prometheus-k8s-db\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959755 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-662mj\" (UniqueName: \"kubernetes.io/projected/baea7031-37dc-491f-97d1-c9efc7bd20f7-kube-api-access-662mj\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959764 2572 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-kube-rbac-proxy\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:11.959790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:11.959790 2572 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/baea7031-37dc-491f-97d1-c9efc7bd20f7-secret-grpc-tls\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:12:12.498730 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498696 2572 generic.go:356] "Generic (PLEG): container finished" podID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" exitCode=0 Jul 01 18:12:12.498730 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498720 2572 generic.go:356] "Generic (PLEG): container finished" podID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" exitCode=0 Jul 01 18:12:12.498730 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498728 2572 generic.go:356] "Generic (PLEG): container finished" podID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" exitCode=0 Jul 01 18:12:12.498730 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498734 2572 generic.go:356] "Generic (PLEG): container finished" podID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" exitCode=0 Jul 01 18:12:12.498730 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498739 2572 generic.go:356] "Generic (PLEG): container finished" podID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" exitCode=0 Jul 01 18:12:12.498730 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498744 2572 generic.go:356] "Generic (PLEG): container finished" podID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" exitCode=0 Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498784 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerDied","Data":"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726"} Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498800 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498846 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerDied","Data":"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26"} Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498873 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerDied","Data":"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c"} Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498900 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerDied","Data":"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797"} Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498925 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerDied","Data":"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27"} Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498930 2572 scope.go:117] "RemoveContainer" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerDied","Data":"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d"} Jul 01 18:12:12.499228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.498971 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"baea7031-37dc-491f-97d1-c9efc7bd20f7","Type":"ContainerDied","Data":"19edd48bb9ab5217d5e21a6202d63696ee7c639bbdf15bcf741f71c9739ec8d4"} Jul 01 18:12:12.507525 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.507407 2572 scope.go:117] "RemoveContainer" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" Jul 01 18:12:12.514457 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.514444 2572 scope.go:117] "RemoveContainer" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" Jul 01 18:12:12.528985 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.528958 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:12:12.534694 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.534675 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:12:12.536852 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.536834 2572 scope.go:117] "RemoveContainer" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" Jul 01 18:12:12.543553 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.543531 2572 scope.go:117] "RemoveContainer" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" Jul 01 18:12:12.549815 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.549800 2572 scope.go:117] "RemoveContainer" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" Jul 01 18:12:12.556414 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.556400 2572 scope.go:117] "RemoveContainer" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" Jul 01 18:12:12.562914 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.562895 2572 scope.go:117] "RemoveContainer" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" Jul 01 18:12:12.563175 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:12:12.563149 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": container with ID starting with 940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726 not found: ID does not exist" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" Jul 01 18:12:12.563232 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.563182 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726"} err="failed to get container status \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": rpc error: code = NotFound desc = could not find container \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": container with ID starting with 940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726 not found: ID does not exist" Jul 01 18:12:12.563232 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.563213 2572 scope.go:117] "RemoveContainer" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" Jul 01 18:12:12.563442 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:12:12.563423 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": container with ID starting with bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26 not found: ID does not exist" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" Jul 01 18:12:12.563491 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.563446 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26"} err="failed to get container status \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": rpc error: code = NotFound desc = could not find container \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": container with ID starting with bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26 not found: ID does not exist" Jul 01 18:12:12.563491 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.563459 2572 scope.go:117] "RemoveContainer" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" Jul 01 18:12:12.563645 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:12:12.563632 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": container with ID starting with d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c not found: ID does not exist" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" Jul 01 18:12:12.563682 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.563656 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c"} err="failed to get container status \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": rpc error: code = NotFound desc = could not find container \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": container with ID starting with d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c not found: ID does not exist" Jul 01 18:12:12.563682 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.563667 2572 scope.go:117] "RemoveContainer" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" Jul 01 18:12:12.563923 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:12:12.563904 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": container with ID starting with 6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797 not found: ID does not exist" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" Jul 01 18:12:12.564021 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.563927 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797"} err="failed to get container status \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": rpc error: code = NotFound desc = could not find container \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": container with ID starting with 6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797 not found: ID does not exist" Jul 01 18:12:12.564021 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.563940 2572 scope.go:117] "RemoveContainer" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" Jul 01 18:12:12.564192 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:12:12.564173 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": container with ID starting with d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27 not found: ID does not exist" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" Jul 01 18:12:12.564253 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.564202 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27"} err="failed to get container status \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": rpc error: code = NotFound desc = could not find container \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": container with ID starting with d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27 not found: ID does not exist" Jul 01 18:12:12.564253 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.564223 2572 scope.go:117] "RemoveContainer" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" Jul 01 18:12:12.564463 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:12:12.564446 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": container with ID starting with 6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d not found: ID does not exist" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" Jul 01 18:12:12.564500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.564468 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d"} err="failed to get container status \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": rpc error: code = NotFound desc = could not find container \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": container with ID starting with 6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d not found: ID does not exist" Jul 01 18:12:12.564500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.564482 2572 scope.go:117] "RemoveContainer" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" Jul 01 18:12:12.564698 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:12:12.564685 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": container with ID starting with 4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982 not found: ID does not exist" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" Jul 01 18:12:12.564739 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.564700 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982"} err="failed to get container status \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": rpc error: code = NotFound desc = could not find container \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": container with ID starting with 4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982 not found: ID does not exist" Jul 01 18:12:12.564739 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.564711 2572 scope.go:117] "RemoveContainer" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" Jul 01 18:12:12.564888 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.564874 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726"} err="failed to get container status \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": rpc error: code = NotFound desc = could not find container \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": container with ID starting with 940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726 not found: ID does not exist" Jul 01 18:12:12.564934 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.564888 2572 scope.go:117] "RemoveContainer" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" Jul 01 18:12:12.565104 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565087 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26"} err="failed to get container status \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": rpc error: code = NotFound desc = could not find container \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": container with ID starting with bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26 not found: ID does not exist" Jul 01 18:12:12.565149 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565105 2572 scope.go:117] "RemoveContainer" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" Jul 01 18:12:12.565326 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565307 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c"} err="failed to get container status \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": rpc error: code = NotFound desc = could not find container \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": container with ID starting with d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c not found: ID does not exist" Jul 01 18:12:12.565373 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565327 2572 scope.go:117] "RemoveContainer" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" Jul 01 18:12:12.565500 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565486 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797"} err="failed to get container status \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": rpc error: code = NotFound desc = could not find container \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": container with ID starting with 6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797 not found: ID does not exist" Jul 01 18:12:12.565540 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565500 2572 scope.go:117] "RemoveContainer" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" Jul 01 18:12:12.565717 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565701 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27"} err="failed to get container status \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": rpc error: code = NotFound desc = could not find container \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": container with ID starting with d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27 not found: ID does not exist" Jul 01 18:12:12.565832 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565717 2572 scope.go:117] "RemoveContainer" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" Jul 01 18:12:12.565953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565932 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d"} err="failed to get container status \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": rpc error: code = NotFound desc = could not find container \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": container with ID starting with 6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d not found: ID does not exist" Jul 01 18:12:12.565998 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.565956 2572 scope.go:117] "RemoveContainer" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" Jul 01 18:12:12.566184 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.566168 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982"} err="failed to get container status \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": rpc error: code = NotFound desc = could not find container \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": container with ID starting with 4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982 not found: ID does not exist" Jul 01 18:12:12.566227 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.566186 2572 scope.go:117] "RemoveContainer" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" Jul 01 18:12:12.566397 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.566380 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726"} err="failed to get container status \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": rpc error: code = NotFound desc = could not find container \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": container with ID starting with 940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726 not found: ID does not exist" Jul 01 18:12:12.566446 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.566398 2572 scope.go:117] "RemoveContainer" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" Jul 01 18:12:12.566627 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.566608 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26"} err="failed to get container status \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": rpc error: code = NotFound desc = could not find container \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": container with ID starting with bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26 not found: ID does not exist" Jul 01 18:12:12.566668 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.566628 2572 scope.go:117] "RemoveContainer" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" Jul 01 18:12:12.566869 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.566852 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c"} err="failed to get container status \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": rpc error: code = NotFound desc = could not find container \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": container with ID starting with d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c not found: ID does not exist" Jul 01 18:12:12.566869 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.566869 2572 scope.go:117] "RemoveContainer" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" Jul 01 18:12:12.567089 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567071 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797"} err="failed to get container status \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": rpc error: code = NotFound desc = could not find container \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": container with ID starting with 6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797 not found: ID does not exist" Jul 01 18:12:12.567132 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567090 2572 scope.go:117] "RemoveContainer" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" Jul 01 18:12:12.567317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567301 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27"} err="failed to get container status \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": rpc error: code = NotFound desc = could not find container \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": container with ID starting with d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27 not found: ID does not exist" Jul 01 18:12:12.567364 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567317 2572 scope.go:117] "RemoveContainer" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" Jul 01 18:12:12.567511 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567495 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d"} err="failed to get container status \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": rpc error: code = NotFound desc = could not find container \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": container with ID starting with 6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d not found: ID does not exist" Jul 01 18:12:12.567547 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567511 2572 scope.go:117] "RemoveContainer" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" Jul 01 18:12:12.567724 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567699 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982"} err="failed to get container status \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": rpc error: code = NotFound desc = could not find container \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": container with ID starting with 4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982 not found: ID does not exist" Jul 01 18:12:12.567724 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567724 2572 scope.go:117] "RemoveContainer" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" Jul 01 18:12:12.567968 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567950 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726"} err="failed to get container status \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": rpc error: code = NotFound desc = could not find container \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": container with ID starting with 940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726 not found: ID does not exist" Jul 01 18:12:12.568026 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.567969 2572 scope.go:117] "RemoveContainer" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" Jul 01 18:12:12.568161 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568146 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26"} err="failed to get container status \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": rpc error: code = NotFound desc = could not find container \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": container with ID starting with bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26 not found: ID does not exist" Jul 01 18:12:12.568212 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568162 2572 scope.go:117] "RemoveContainer" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" Jul 01 18:12:12.568380 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568361 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c"} err="failed to get container status \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": rpc error: code = NotFound desc = could not find container \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": container with ID starting with d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c not found: ID does not exist" Jul 01 18:12:12.568432 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568381 2572 scope.go:117] "RemoveContainer" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" Jul 01 18:12:12.568580 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568563 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797"} err="failed to get container status \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": rpc error: code = NotFound desc = could not find container \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": container with ID starting with 6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797 not found: ID does not exist" Jul 01 18:12:12.568631 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568580 2572 scope.go:117] "RemoveContainer" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" Jul 01 18:12:12.568780 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568751 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27"} err="failed to get container status \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": rpc error: code = NotFound desc = could not find container \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": container with ID starting with d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27 not found: ID does not exist" Jul 01 18:12:12.568822 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568783 2572 scope.go:117] "RemoveContainer" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" Jul 01 18:12:12.569003 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.568986 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d"} err="failed to get container status \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": rpc error: code = NotFound desc = could not find container \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": container with ID starting with 6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d not found: ID does not exist" Jul 01 18:12:12.569050 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569003 2572 scope.go:117] "RemoveContainer" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" Jul 01 18:12:12.569225 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569209 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982"} err="failed to get container status \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": rpc error: code = NotFound desc = could not find container \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": container with ID starting with 4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982 not found: ID does not exist" Jul 01 18:12:12.569225 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569224 2572 scope.go:117] "RemoveContainer" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" Jul 01 18:12:12.569423 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569407 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726"} err="failed to get container status \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": rpc error: code = NotFound desc = could not find container \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": container with ID starting with 940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726 not found: ID does not exist" Jul 01 18:12:12.569423 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569423 2572 scope.go:117] "RemoveContainer" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" Jul 01 18:12:12.569625 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569609 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26"} err="failed to get container status \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": rpc error: code = NotFound desc = could not find container \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": container with ID starting with bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26 not found: ID does not exist" Jul 01 18:12:12.569673 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569625 2572 scope.go:117] "RemoveContainer" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" Jul 01 18:12:12.569843 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569821 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c"} err="failed to get container status \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": rpc error: code = NotFound desc = could not find container \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": container with ID starting with d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c not found: ID does not exist" Jul 01 18:12:12.569843 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.569841 2572 scope.go:117] "RemoveContainer" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" Jul 01 18:12:12.570042 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570023 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797"} err="failed to get container status \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": rpc error: code = NotFound desc = could not find container \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": container with ID starting with 6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797 not found: ID does not exist" Jul 01 18:12:12.570087 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570043 2572 scope.go:117] "RemoveContainer" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" Jul 01 18:12:12.570264 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570246 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27"} err="failed to get container status \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": rpc error: code = NotFound desc = could not find container \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": container with ID starting with d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27 not found: ID does not exist" Jul 01 18:12:12.570311 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570265 2572 scope.go:117] "RemoveContainer" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" Jul 01 18:12:12.570455 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570441 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d"} err="failed to get container status \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": rpc error: code = NotFound desc = could not find container \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": container with ID starting with 6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d not found: ID does not exist" Jul 01 18:12:12.570497 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570458 2572 scope.go:117] "RemoveContainer" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" Jul 01 18:12:12.570679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570663 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982"} err="failed to get container status \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": rpc error: code = NotFound desc = could not find container \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": container with ID starting with 4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982 not found: ID does not exist" Jul 01 18:12:12.570719 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570679 2572 scope.go:117] "RemoveContainer" containerID="940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726" Jul 01 18:12:12.570871 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570855 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726"} err="failed to get container status \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": rpc error: code = NotFound desc = could not find container \"940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726\": container with ID starting with 940d406ceefc9b681308225fa805d27e74501c17b1dc0640bf3c67ef820c3726 not found: ID does not exist" Jul 01 18:12:12.570927 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.570871 2572 scope.go:117] "RemoveContainer" containerID="bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26" Jul 01 18:12:12.571090 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571074 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26"} err="failed to get container status \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": rpc error: code = NotFound desc = could not find container \"bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26\": container with ID starting with bc8bbaba63077ee71451fcc9e41e9b682156dbca9e7b0999656fc6500ed00c26 not found: ID does not exist" Jul 01 18:12:12.571134 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571091 2572 scope.go:117] "RemoveContainer" containerID="d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c" Jul 01 18:12:12.571311 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571293 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c"} err="failed to get container status \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": rpc error: code = NotFound desc = could not find container \"d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c\": container with ID starting with d7d9bfc9a3f91d4efc7f12e963ccd8a844e5fc3bd9a779e27024d49d6ae5036c not found: ID does not exist" Jul 01 18:12:12.571352 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571311 2572 scope.go:117] "RemoveContainer" containerID="6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797" Jul 01 18:12:12.571520 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571504 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797"} err="failed to get container status \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": rpc error: code = NotFound desc = could not find container \"6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797\": container with ID starting with 6e072dda02429affc5f7f6550539660bcb23229c40acd7855becb461dea59797 not found: ID does not exist" Jul 01 18:12:12.571520 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571519 2572 scope.go:117] "RemoveContainer" containerID="d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27" Jul 01 18:12:12.571716 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571700 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27"} err="failed to get container status \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": rpc error: code = NotFound desc = could not find container \"d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27\": container with ID starting with d92a19385e1abc6684527b2dad8c1db0bcd1a3b02394ebf4b87b63a920223d27 not found: ID does not exist" Jul 01 18:12:12.571759 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571716 2572 scope.go:117] "RemoveContainer" containerID="6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d" Jul 01 18:12:12.571996 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571929 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d"} err="failed to get container status \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": rpc error: code = NotFound desc = could not find container \"6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d\": container with ID starting with 6b1d64f503672aff852836d237c1013d03039948b346e4732ea4606e5e3ed94d not found: ID does not exist" Jul 01 18:12:12.571996 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.571949 2572 scope.go:117] "RemoveContainer" containerID="4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982" Jul 01 18:12:12.572147 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.572128 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982"} err="failed to get container status \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": rpc error: code = NotFound desc = could not find container \"4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982\": container with ID starting with 4de95f5334ee0b2f4ee8b5eb06563cd90677e38f3436c5dda822220c73afb982 not found: ID does not exist" Jul 01 18:12:12.592588 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592570 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:12:12.592903 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592890 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="init-config-reloader" Jul 01 18:12:12.592953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592905 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="init-config-reloader" Jul 01 18:12:12.592953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592924 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="config-reloader" Jul 01 18:12:12.592953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592930 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="config-reloader" Jul 01 18:12:12.592953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592937 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="prometheus" Jul 01 18:12:12.592953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592942 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="prometheus" Jul 01 18:12:12.592953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592948 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy-web" Jul 01 18:12:12.592953 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592953 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy-web" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592961 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="thanos-sidecar" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592967 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="thanos-sidecar" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592972 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy-thanos" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592977 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy-thanos" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592985 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.592989 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.593030 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="thanos-sidecar" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.593038 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy-thanos" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.593044 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.593050 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="prometheus" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.593056 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="config-reloader" Jul 01 18:12:12.593197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.593062 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" containerName="kube-rbac-proxy-web" Jul 01 18:12:12.596710 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.596697 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.598951 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.598921 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 18:12:12.599033 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.599015 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 18:12:12.599321 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.599307 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 18:12:12.599491 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.599474 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 18:12:12.599583 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.599507 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m7fzl\"" Jul 01 18:12:12.599645 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.599588 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 18:12:12.599834 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.599820 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 18:12:12.599971 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.599950 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 18:12:12.600104 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.600010 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 18:12:12.600237 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.600198 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6l3kpjv945fnf\"" Jul 01 18:12:12.600296 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.600262 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 18:12:12.600296 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.600262 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 18:12:12.600433 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.600300 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 18:12:12.600433 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.600304 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 18:12:12.600433 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.600427 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 18:12:12.604214 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.604196 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 18:12:12.605578 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.605559 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 18:12:12.617498 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.617476 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:12:12.665124 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665098 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665209 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665128 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c184770e-e529-4cf1-aa14-b45f6c41bf64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665209 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665145 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665209 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665167 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665322 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665218 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665322 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665238 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-config\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665322 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665295 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665322 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665320 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665436 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665343 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665436 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665378 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665436 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665395 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665436 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665409 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665548 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665449 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665548 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665467 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c184770e-e529-4cf1-aa14-b45f6c41bf64-config-out\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665548 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665493 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-web-config\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665548 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665511 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665548 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665536 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7jmg\" (UniqueName: \"kubernetes.io/projected/c184770e-e529-4cf1-aa14-b45f6c41bf64-kube-api-access-z7jmg\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665684 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665557 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665684 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665578 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.665684 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.665600 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.766669 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766602 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.766669 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766645 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.766669 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766669 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.766893 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.766893 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766720 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.766893 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766745 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c184770e-e529-4cf1-aa14-b45f6c41bf64-config-out\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.766893 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766810 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-web-config\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767087 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766964 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767087 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.766999 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7jmg\" (UniqueName: \"kubernetes.io/projected/c184770e-e529-4cf1-aa14-b45f6c41bf64-kube-api-access-z7jmg\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767087 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767034 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767087 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767064 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767284 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767091 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767284 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767124 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767284 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767154 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c184770e-e529-4cf1-aa14-b45f6c41bf64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767284 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767179 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767284 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767284 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767267 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767571 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767293 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-config\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767571 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767318 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767571 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767345 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767571 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767364 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.767571 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.767432 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.768107 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.768084 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.768992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.768831 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.770019 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.769523 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c184770e-e529-4cf1-aa14-b45f6c41bf64-config-out\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.770019 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.769724 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.770019 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.769755 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.770019 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.769872 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c184770e-e529-4cf1-aa14-b45f6c41bf64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.770252 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.770043 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.770649 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.770625 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-web-config\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.771230 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.771203 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.771323 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.771239 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.771323 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.771239 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.771674 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.771650 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.772058 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.772035 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c184770e-e529-4cf1-aa14-b45f6c41bf64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.772458 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.772433 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.772791 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.772753 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-config\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.772863 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.772750 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.773574 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.773553 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c184770e-e529-4cf1-aa14-b45f6c41bf64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.778425 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.778405 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7jmg\" (UniqueName: \"kubernetes.io/projected/c184770e-e529-4cf1-aa14-b45f6c41bf64-kube-api-access-z7jmg\") pod \"prometheus-k8s-0\" (UID: \"c184770e-e529-4cf1-aa14-b45f6c41bf64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:12.907475 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:12.907439 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:12:13.046278 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:13.046207 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:12:13.049249 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:12:13.049221 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc184770e_e529_4cf1_aa14_b45f6c41bf64.slice/crio-d9f7c0fba5a524825dc9e39b8257d1e709b3fd9cf679b5080f3cbb0bf179b8f0 WatchSource:0}: Error finding container d9f7c0fba5a524825dc9e39b8257d1e709b3fd9cf679b5080f3cbb0bf179b8f0: Status 404 returned error can't find the container with id d9f7c0fba5a524825dc9e39b8257d1e709b3fd9cf679b5080f3cbb0bf179b8f0 Jul 01 18:12:13.361633 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:13.361562 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baea7031-37dc-491f-97d1-c9efc7bd20f7" path="/var/lib/kubelet/pods/baea7031-37dc-491f-97d1-c9efc7bd20f7/volumes" Jul 01 18:12:13.505059 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:13.505023 2572 generic.go:356] "Generic (PLEG): container finished" podID="c184770e-e529-4cf1-aa14-b45f6c41bf64" containerID="cb0adb5ab11cc0db2a0cb83c39c483b0aefcf4e67979fc2fcfc0bb7c5765463a" exitCode=0 Jul 01 18:12:13.505229 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:13.505107 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c184770e-e529-4cf1-aa14-b45f6c41bf64","Type":"ContainerDied","Data":"cb0adb5ab11cc0db2a0cb83c39c483b0aefcf4e67979fc2fcfc0bb7c5765463a"} Jul 01 18:12:13.505229 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:13.505174 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c184770e-e529-4cf1-aa14-b45f6c41bf64","Type":"ContainerStarted","Data":"d9f7c0fba5a524825dc9e39b8257d1e709b3fd9cf679b5080f3cbb0bf179b8f0"} Jul 01 18:12:14.515423 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:14.515391 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c184770e-e529-4cf1-aa14-b45f6c41bf64","Type":"ContainerStarted","Data":"694b60d0856a4f9eae4d2a20c1a4f1611998962f96819f000dc2593d3ef71fdb"} Jul 01 18:12:14.515787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:14.515480 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c184770e-e529-4cf1-aa14-b45f6c41bf64","Type":"ContainerStarted","Data":"554627618b1e462a746feba598ff8a6f835f573e8cdcbe657eaa839d447eeb41"} Jul 01 18:12:14.515787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:14.515507 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c184770e-e529-4cf1-aa14-b45f6c41bf64","Type":"ContainerStarted","Data":"5a876cb9c0e7445c88631d026a5fcfa89a39a76db7a80bd34634532d966bdeeb"} Jul 01 18:12:14.515787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:14.515534 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c184770e-e529-4cf1-aa14-b45f6c41bf64","Type":"ContainerStarted","Data":"4cdfa0e58aa9a6bc2838b44b4aa852d2e25783937f19b7480c3a686d83283622"} Jul 01 18:12:14.515787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:14.515574 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c184770e-e529-4cf1-aa14-b45f6c41bf64","Type":"ContainerStarted","Data":"d2769f9ddca47b3325d5e42837559b8c1b1b2145f0a29061e5eec149b05f2b26"} Jul 01 18:12:14.515787 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:14.515597 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c184770e-e529-4cf1-aa14-b45f6c41bf64","Type":"ContainerStarted","Data":"15711ccc479b98ef48c2cff8aa10bd358516b9dbf4faaa18d741ea9273d8d4d2"} Jul 01 18:12:14.543095 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:14.543037 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.520203135 podStartE2EDuration="2.54301789s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.814755ms" isStatefulPod=false podCreationTimestamp="2026-07-01 18:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:12:14.541319178 +0000 UTC m=+343.682502550" watchObservedRunningTime="2026-07-01 18:12:14.54301789 +0000 UTC m=+343.684201263" Jul 01 18:12:17.907723 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:12:17.907680 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:13:12.908052 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:13:12.908015 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:13:12.923054 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:13:12.923033 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:13:13.760646 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:13:13.760622 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:17:56.067007 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.066972 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-bdft9"] Jul 01 18:17:56.071977 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.071939 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-bdft9" Jul 01 18:17:56.074667 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.074638 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 01 18:17:56.074667 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.074654 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 01 18:17:56.074885 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.074733 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-l97mm\"" Jul 01 18:17:56.075881 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.075751 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 01 18:17:56.077666 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.077644 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-bdft9"] Jul 01 18:17:56.152938 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.152911 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h94j9\" (UniqueName: \"kubernetes.io/projected/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-kube-api-access-h94j9\") pod \"s3-init-bdft9\" (UID: \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\") " pod="kserve/s3-init-bdft9" Jul 01 18:17:56.153066 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.152951 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-model-cache\") pod \"s3-init-bdft9\" (UID: \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\") " pod="kserve/s3-init-bdft9" Jul 01 18:17:56.254028 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.253992 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h94j9\" (UniqueName: \"kubernetes.io/projected/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-kube-api-access-h94j9\") pod \"s3-init-bdft9\" (UID: \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\") " pod="kserve/s3-init-bdft9" Jul 01 18:17:56.254148 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.254046 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-model-cache\") pod \"s3-init-bdft9\" (UID: \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\") " pod="kserve/s3-init-bdft9" Jul 01 18:17:56.254342 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.254327 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-model-cache\") pod \"s3-init-bdft9\" (UID: \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\") " pod="kserve/s3-init-bdft9" Jul 01 18:17:56.265048 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.265023 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h94j9\" (UniqueName: \"kubernetes.io/projected/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-kube-api-access-h94j9\") pod \"s3-init-bdft9\" (UID: \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\") " pod="kserve/s3-init-bdft9" Jul 01 18:17:56.396857 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.396836 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-bdft9" Jul 01 18:17:56.526679 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.526655 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-bdft9"] Jul 01 18:17:56.529313 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:17:56.529291 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c9c7398_72d3_4dc2_966b_0c1c4a3db4b0.slice/crio-7fd7c1035d9314f77ad8c580b946b2c7eb53307e4e7976630a625bba6a21e06b WatchSource:0}: Error finding container 7fd7c1035d9314f77ad8c580b946b2c7eb53307e4e7976630a625bba6a21e06b: Status 404 returned error can't find the container with id 7fd7c1035d9314f77ad8c580b946b2c7eb53307e4e7976630a625bba6a21e06b Jul 01 18:17:56.531544 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.531524 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 18:17:56.834878 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:17:56.834798 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bdft9" event={"ID":"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0","Type":"ContainerStarted","Data":"7fd7c1035d9314f77ad8c580b946b2c7eb53307e4e7976630a625bba6a21e06b"} Jul 01 18:18:00.853912 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:18:00.853876 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bdft9" event={"ID":"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0","Type":"ContainerStarted","Data":"435ad058d4f513221f1a40f7cce50388c5b31eaff4b240a05b9c40023a22cd3f"} Jul 01 18:18:01.859756 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:18:01.859721 2572 generic.go:356] "Generic (PLEG): container finished" podID="2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" containerID="435ad058d4f513221f1a40f7cce50388c5b31eaff4b240a05b9c40023a22cd3f" exitCode=0 Jul 01 18:18:01.860140 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:18:01.859798 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bdft9" event={"ID":"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0","Type":"ContainerDied","Data":"435ad058d4f513221f1a40f7cce50388c5b31eaff4b240a05b9c40023a22cd3f"} Jul 01 18:18:07.890395 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:18:07.890361 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bdft9" event={"ID":"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0","Type":"ContainerStarted","Data":"34ea6022061434209507ac6accd2ffb9799443e9854726d485ac50a5637e4a22"} Jul 01 18:18:07.908851 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:18:07.908800 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-bdft9" podStartSLOduration=1.120558422 podStartE2EDuration="11.908783104s" totalImagesPullingTime="8.987731805s" totalInitContainerRuntime="1.800492877s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:17:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:18:07.907096849 +0000 UTC m=+697.048280233" watchObservedRunningTime="2026-07-01 18:18:07.908783104 +0000 UTC m=+697.049966467" Jul 01 18:19:13.144136 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:13.144098 2572 generic.go:356] "Generic (PLEG): container finished" podID="2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" containerID="34ea6022061434209507ac6accd2ffb9799443e9854726d485ac50a5637e4a22" exitCode=0 Jul 01 18:19:13.144533 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:13.144136 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bdft9" event={"ID":"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0","Type":"ContainerDied","Data":"34ea6022061434209507ac6accd2ffb9799443e9854726d485ac50a5637e4a22"} Jul 01 18:19:14.267803 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:14.267763 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-bdft9" Jul 01 18:19:14.410631 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:14.410544 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-model-cache\") pod \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\" (UID: \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\") " Jul 01 18:19:14.410795 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:14.410635 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h94j9\" (UniqueName: \"kubernetes.io/projected/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-kube-api-access-h94j9\") pod \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\" (UID: \"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0\") " Jul 01 18:19:14.412777 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:14.412741 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-kube-api-access-h94j9" (OuterVolumeSpecName: "kube-api-access-h94j9") pod "2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" (UID: "2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0"). InnerVolumeSpecName "kube-api-access-h94j9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:19:14.429228 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:14.429203 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" (UID: "2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:19:14.511614 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:14.511587 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h94j9\" (UniqueName: \"kubernetes.io/projected/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-kube-api-access-h94j9\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:19:14.511614 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:14.511612 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0-model-cache\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:19:15.153066 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:15.153041 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-bdft9" Jul 01 18:19:15.153244 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:15.153069 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-bdft9" event={"ID":"2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0","Type":"ContainerDied","Data":"7fd7c1035d9314f77ad8c580b946b2c7eb53307e4e7976630a625bba6a21e06b"} Jul 01 18:19:15.153244 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:15.153098 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fd7c1035d9314f77ad8c580b946b2c7eb53307e4e7976630a625bba6a21e06b" Jul 01 18:19:27.855941 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.855909 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q"] Jul 01 18:19:27.856314 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.856240 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" containerName="download-hf-model" Jul 01 18:19:27.856314 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.856253 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" containerName="download-hf-model" Jul 01 18:19:27.856314 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.856274 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" containerName="s3-init" Jul 01 18:19:27.856314 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.856279 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" containerName="s3-init" Jul 01 18:19:27.856439 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.856330 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0" containerName="s3-init" Jul 01 18:19:27.859385 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.859369 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:27.861707 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.861686 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 01 18:19:27.861842 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.861742 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 01 18:19:27.861935 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.861915 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 01 18:19:27.862051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.861934 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 01 18:19:27.862051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.861991 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 18:19:27.862866 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.862849 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 01 18:19:27.862936 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.862850 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-fg8rn\"" Jul 01 18:19:27.871091 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:27.871067 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q"] Jul 01 18:19:28.033904 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.033872 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.034073 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.033920 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.034073 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.033946 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sglr\" (UniqueName: \"kubernetes.io/projected/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-api-access-2sglr\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.034073 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.033997 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.034073 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.034057 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/58761e7f-5592-4b8d-a9c8-876f55a83277-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.068172 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.068137 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d"] Jul 01 18:19:28.071568 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.071553 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.073895 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.073873 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 01 18:19:28.074014 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.073894 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 01 18:19:28.081574 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.081552 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d"] Jul 01 18:19:28.135051 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135025 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.135197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135064 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.135197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.135197 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135107 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.135291 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.135291 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sglr\" (UniqueName: \"kubernetes.io/projected/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-api-access-2sglr\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.135291 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135272 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.135408 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135294 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.135408 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135330 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6xjn\" (UniqueName: \"kubernetes.io/projected/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-api-access-s6xjn\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.135408 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135370 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/58761e7f-5592-4b8d-a9c8-876f55a83277-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.135535 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:19:28.135422 2572 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Jul 01 18:19:28.135535 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:19:28.135503 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls podName:58761e7f-5592-4b8d-a9c8-876f55a83277 nodeName:}" failed. No retries permitted until 2026-07-01 18:19:28.63548201 +0000 UTC m=+777.776665372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls") pod "isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" (UID: "58761e7f-5592-4b8d-a9c8-876f55a83277") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Jul 01 18:19:28.135688 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135672 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/58761e7f-5592-4b8d-a9c8-876f55a83277-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.135819 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135800 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.135852 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.135805 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.145788 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.145754 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sglr\" (UniqueName: \"kubernetes.io/projected/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-api-access-2sglr\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.223294 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.223263 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf"] Jul 01 18:19:28.226951 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.226935 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.229246 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.229230 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 01 18:19:28.229296 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.229234 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 01 18:19:28.236070 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236044 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.236198 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236102 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b070bf96-654c-4b52-95dd-df1b1be88642-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.236198 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236140 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6xjn\" (UniqueName: \"kubernetes.io/projected/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-api-access-s6xjn\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.236198 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236170 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.236339 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:19:28.236193 2572 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found Jul 01 18:19:28.236339 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236211 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.236339 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236241 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld87k\" (UniqueName: \"kubernetes.io/projected/b070bf96-654c-4b52-95dd-df1b1be88642-kube-api-access-ld87k\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.236339 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:19:28.236262 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls podName:20a33dae-136c-4a92-8a1c-e51e0ca7a640 nodeName:}" failed. No retries permitted until 2026-07-01 18:19:28.736244496 +0000 UTC m=+777.877427853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls") pod "isvc-xgboost-graph-predictor-555fc69d56-hcw7d" (UID: "20a33dae-136c-4a92-8a1c-e51e0ca7a640") : secret "isvc-xgboost-graph-predictor-serving-cert" not found Jul 01 18:19:28.236339 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236321 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b070bf96-654c-4b52-95dd-df1b1be88642-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.236569 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236382 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.236569 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236466 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.236569 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236495 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.236759 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236735 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.236957 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.236940 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf"] Jul 01 18:19:28.237231 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.237208 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.237317 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.237208 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.247530 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.247507 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6xjn\" (UniqueName: \"kubernetes.io/projected/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-api-access-s6xjn\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.336920 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.336882 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.337101 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.336927 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.337101 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.336953 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld87k\" (UniqueName: \"kubernetes.io/projected/b070bf96-654c-4b52-95dd-df1b1be88642-kube-api-access-ld87k\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.337101 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.336972 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b070bf96-654c-4b52-95dd-df1b1be88642-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.337101 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.337056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b070bf96-654c-4b52-95dd-df1b1be88642-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.337451 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.337432 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b070bf96-654c-4b52-95dd-df1b1be88642-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.337575 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.337554 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.337701 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.337681 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.339511 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.339489 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b070bf96-654c-4b52-95dd-df1b1be88642-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.344719 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.344699 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld87k\" (UniqueName: \"kubernetes.io/projected/b070bf96-654c-4b52-95dd-df1b1be88642-kube-api-access-ld87k\") pod \"isvc-sklearn-graph-2-predictor-84b568d556-vxgvf\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.538532 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.538441 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:19:28.639231 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.639189 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.641508 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.641488 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.669088 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.669057 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf"] Jul 01 18:19:28.673000 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:19:28.672972 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb070bf96_654c_4b52_95dd_df1b1be88642.slice/crio-8fdf97b787d9d7a54f15382f974dd20b5d493aa38ab70384f4e2c8801c013bc5 WatchSource:0}: Error finding container 8fdf97b787d9d7a54f15382f974dd20b5d493aa38ab70384f4e2c8801c013bc5: Status 404 returned error can't find the container with id 8fdf97b787d9d7a54f15382f974dd20b5d493aa38ab70384f4e2c8801c013bc5 Jul 01 18:19:28.740545 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.740514 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.742551 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.742529 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-hcw7d\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:28.770054 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.770022 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:19:28.904033 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.904008 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q"] Jul 01 18:19:28.906880 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:19:28.906855 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58761e7f_5592_4b8d_a9c8_876f55a83277.slice/crio-0918ecf1be23b10f502a15c8fd24947b1a39778611e3ffb2ec6de5d38f8cd6a5 WatchSource:0}: Error finding container 0918ecf1be23b10f502a15c8fd24947b1a39778611e3ffb2ec6de5d38f8cd6a5: Status 404 returned error can't find the container with id 0918ecf1be23b10f502a15c8fd24947b1a39778611e3ffb2ec6de5d38f8cd6a5 Jul 01 18:19:28.982156 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:28.982132 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:19:29.122240 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:29.122209 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d"] Jul 01 18:19:29.126446 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:19:29.126421 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a33dae_136c_4a92_8a1c_e51e0ca7a640.slice/crio-313e9fb2b23c4d25cf24d04e51d484b0938aaab51995a8defb0d1cd37da1ae59 WatchSource:0}: Error finding container 313e9fb2b23c4d25cf24d04e51d484b0938aaab51995a8defb0d1cd37da1ae59: Status 404 returned error can't find the container with id 313e9fb2b23c4d25cf24d04e51d484b0938aaab51995a8defb0d1cd37da1ae59 Jul 01 18:19:29.211531 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:29.211502 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" event={"ID":"58761e7f-5592-4b8d-a9c8-876f55a83277","Type":"ContainerStarted","Data":"afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179"} Jul 01 18:19:29.211660 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:29.211570 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" event={"ID":"58761e7f-5592-4b8d-a9c8-876f55a83277","Type":"ContainerStarted","Data":"0918ecf1be23b10f502a15c8fd24947b1a39778611e3ffb2ec6de5d38f8cd6a5"} Jul 01 18:19:29.213429 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:29.213384 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" event={"ID":"b070bf96-654c-4b52-95dd-df1b1be88642","Type":"ContainerStarted","Data":"06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34"} Jul 01 18:19:29.213534 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:29.213445 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" event={"ID":"b070bf96-654c-4b52-95dd-df1b1be88642","Type":"ContainerStarted","Data":"8fdf97b787d9d7a54f15382f974dd20b5d493aa38ab70384f4e2c8801c013bc5"} Jul 01 18:19:29.215206 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:29.215181 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" event={"ID":"20a33dae-136c-4a92-8a1c-e51e0ca7a640","Type":"ContainerStarted","Data":"7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f"} Jul 01 18:19:29.215297 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:29.215239 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" event={"ID":"20a33dae-136c-4a92-8a1c-e51e0ca7a640","Type":"ContainerStarted","Data":"313e9fb2b23c4d25cf24d04e51d484b0938aaab51995a8defb0d1cd37da1ae59"} Jul 01 18:19:33.232993 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:33.232905 2572 generic.go:356] "Generic (PLEG): container finished" podID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerID="afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179" exitCode=0 Jul 01 18:19:33.232993 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:33.232966 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" event={"ID":"58761e7f-5592-4b8d-a9c8-876f55a83277","Type":"ContainerDied","Data":"afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179"} Jul 01 18:19:33.234532 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:33.234506 2572 generic.go:356] "Generic (PLEG): container finished" podID="b070bf96-654c-4b52-95dd-df1b1be88642" containerID="06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34" exitCode=0 Jul 01 18:19:33.234641 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:33.234551 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" event={"ID":"b070bf96-654c-4b52-95dd-df1b1be88642","Type":"ContainerDied","Data":"06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34"} Jul 01 18:19:33.236191 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:33.236175 2572 generic.go:356] "Generic (PLEG): container finished" podID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerID="7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f" exitCode=0 Jul 01 18:19:33.236263 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:19:33.236199 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" event={"ID":"20a33dae-136c-4a92-8a1c-e51e0ca7a640","Type":"ContainerDied","Data":"7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f"} Jul 01 18:20:01.429033 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:01.429000 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" event={"ID":"58761e7f-5592-4b8d-a9c8-876f55a83277","Type":"ContainerStarted","Data":"a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2"} Jul 01 18:20:01.430735 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:01.430710 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" event={"ID":"b070bf96-654c-4b52-95dd-df1b1be88642","Type":"ContainerStarted","Data":"8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b"} Jul 01 18:20:01.432377 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:01.432360 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" event={"ID":"20a33dae-136c-4a92-8a1c-e51e0ca7a640","Type":"ContainerStarted","Data":"2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854"} Jul 01 18:20:05.458025 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.457989 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" event={"ID":"58761e7f-5592-4b8d-a9c8-876f55a83277","Type":"ContainerStarted","Data":"b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a"} Jul 01 18:20:05.458521 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.458146 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:20:05.461175 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.461142 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" event={"ID":"b070bf96-654c-4b52-95dd-df1b1be88642","Type":"ContainerStarted","Data":"917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2"} Jul 01 18:20:05.461307 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.461215 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:20:05.461307 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.461250 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:20:05.463543 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.463505 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:20:05.464081 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.464057 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" event={"ID":"20a33dae-136c-4a92-8a1c-e51e0ca7a640","Type":"ContainerStarted","Data":"a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5"} Jul 01 18:20:05.464312 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.464296 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:20:05.464386 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.464339 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:20:05.465331 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.465307 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:20:05.478492 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.478443 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podStartSLOduration=3.57709717 podStartE2EDuration="38.478426792s" totalImagesPullingTime="31.33645931s" totalInitContainerRuntime="3.564870312s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:19:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:20:05.477561301 +0000 UTC m=+814.618744673" watchObservedRunningTime="2026-07-01 18:20:05.478426792 +0000 UTC m=+814.619610164" Jul 01 18:20:05.494669 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.494623 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podStartSLOduration=2.253166339 podStartE2EDuration="37.49460894s" totalImagesPullingTime="31.428426733s" totalInitContainerRuntime="3.813015868s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:19:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:20:05.49325931 +0000 UTC m=+814.634442713" watchObservedRunningTime="2026-07-01 18:20:05.49460894 +0000 UTC m=+814.635792311" Jul 01 18:20:05.511321 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:05.511275 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podStartSLOduration=2.400518286 podStartE2EDuration="37.511260488s" totalImagesPullingTime="31.388918786s" totalInitContainerRuntime="3.721823416s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:19:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:20:05.509129821 +0000 UTC m=+814.650313203" watchObservedRunningTime="2026-07-01 18:20:05.511260488 +0000 UTC m=+814.652443859" Jul 01 18:20:06.469096 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:06.469060 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:20:06.469096 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:06.469087 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:20:06.469633 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:06.469379 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:20:06.470207 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:06.470184 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:20:07.473455 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:07.473408 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:20:11.474992 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:11.474957 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:20:11.475339 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:11.475042 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:20:11.475529 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:11.475493 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:20:11.475597 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:11.475562 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:20:12.479239 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:12.479213 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:20:12.479730 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:12.479703 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:20:21.475846 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:21.475766 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:20:21.476232 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:21.475784 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:20:22.480277 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:22.480236 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:20:31.475991 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:31.475957 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:20:31.476363 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:31.475959 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:20:32.480567 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:32.480527 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:20:41.476011 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:41.475972 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:20:41.476396 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:41.475973 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:20:42.480621 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:42.480583 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:20:51.475951 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:51.475903 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:20:51.475951 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:51.475922 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:20:52.479840 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:20:52.479800 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:21:01.475804 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:01.475745 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:21:01.476181 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:01.475754 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:21:02.479765 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:02.479728 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:21:11.475673 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:11.475634 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:21:11.476324 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:11.475890 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:21:12.479752 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:12.479717 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:21:21.476918 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:21.476892 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:21:22.480995 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:22.480927 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:21:57.831131 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.831099 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d"] Jul 01 18:21:57.831688 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.831342 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" containerID="cri-o://2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854" gracePeriod=30 Jul 01 18:21:57.831688 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.831407 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kube-rbac-proxy" containerID="cri-o://a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5" gracePeriod=30 Jul 01 18:21:57.885015 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.884985 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf"] Jul 01 18:21:57.885297 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.885247 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" containerID="cri-o://8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b" gracePeriod=30 Jul 01 18:21:57.885437 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.885320 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kube-rbac-proxy" containerID="cri-o://917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2" gracePeriod=30 Jul 01 18:21:57.939722 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.939692 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q"] Jul 01 18:21:57.940019 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.939979 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" containerID="cri-o://a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2" gracePeriod=30 Jul 01 18:21:57.940144 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.939988 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kube-rbac-proxy" containerID="cri-o://b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a" gracePeriod=30 Jul 01 18:21:57.946943 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.946921 2572 generic.go:356] "Generic (PLEG): container finished" podID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerID="a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5" exitCode=2 Jul 01 18:21:57.947039 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:57.946971 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" event={"ID":"20a33dae-136c-4a92-8a1c-e51e0ca7a640","Type":"ContainerDied","Data":"a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5"} Jul 01 18:21:58.953966 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:58.953930 2572 generic.go:356] "Generic (PLEG): container finished" podID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerID="b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a" exitCode=2 Jul 01 18:21:58.954422 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:58.954001 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" event={"ID":"58761e7f-5592-4b8d-a9c8-876f55a83277","Type":"ContainerDied","Data":"b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a"} Jul 01 18:21:58.956413 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:58.956387 2572 generic.go:356] "Generic (PLEG): container finished" podID="b070bf96-654c-4b52-95dd-df1b1be88642" containerID="917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2" exitCode=2 Jul 01 18:21:58.956539 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:21:58.956467 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" event={"ID":"b070bf96-654c-4b52-95dd-df1b1be88642","Type":"ContainerDied","Data":"917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2"} Jul 01 18:22:01.470143 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.470104 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused" Jul 01 18:22:01.470494 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.470112 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.23:8643/healthz\": dial tcp 10.133.0.23:8643: connect: connection refused" Jul 01 18:22:01.475490 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.475462 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused" Jul 01 18:22:01.475657 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.475637 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 18:22:01.574388 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.574363 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:22:01.595404 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.595375 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls\") pod \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " Jul 01 18:22:01.595538 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.595445 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-rbac-proxy-sar-config\") pod \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " Jul 01 18:22:01.595538 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.595483 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kserve-provision-location\") pod \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " Jul 01 18:22:01.595538 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.595512 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-cabundle-cert\") pod \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " Jul 01 18:22:01.595706 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.595559 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6xjn\" (UniqueName: \"kubernetes.io/projected/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-api-access-s6xjn\") pod \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\" (UID: \"20a33dae-136c-4a92-8a1c-e51e0ca7a640\") " Jul 01 18:22:01.595895 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.595865 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "20a33dae-136c-4a92-8a1c-e51e0ca7a640" (UID: "20a33dae-136c-4a92-8a1c-e51e0ca7a640"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:22:01.596004 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.595903 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "20a33dae-136c-4a92-8a1c-e51e0ca7a640" (UID: "20a33dae-136c-4a92-8a1c-e51e0ca7a640"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:22:01.596004 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.595922 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "20a33dae-136c-4a92-8a1c-e51e0ca7a640" (UID: "20a33dae-136c-4a92-8a1c-e51e0ca7a640"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:22:01.597585 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.597557 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-api-access-s6xjn" (OuterVolumeSpecName: "kube-api-access-s6xjn") pod "20a33dae-136c-4a92-8a1c-e51e0ca7a640" (UID: "20a33dae-136c-4a92-8a1c-e51e0ca7a640"). InnerVolumeSpecName "kube-api-access-s6xjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:22:01.597692 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.597624 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "20a33dae-136c-4a92-8a1c-e51e0ca7a640" (UID: "20a33dae-136c-4a92-8a1c-e51e0ca7a640"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:22:01.696552 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.696529 2572 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-cabundle-cert\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:01.696552 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.696550 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6xjn\" (UniqueName: \"kubernetes.io/projected/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-api-access-s6xjn\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:01.696722 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.696562 2572 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/20a33dae-136c-4a92-8a1c-e51e0ca7a640-proxy-tls\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:01.696722 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.696570 2572 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:01.696722 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.696578 2572 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20a33dae-136c-4a92-8a1c-e51e0ca7a640-kserve-provision-location\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:01.971746 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.971655 2572 generic.go:356] "Generic (PLEG): container finished" podID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerID="2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854" exitCode=0 Jul 01 18:22:01.971746 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.971713 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" Jul 01 18:22:01.971966 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.971745 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" event={"ID":"20a33dae-136c-4a92-8a1c-e51e0ca7a640","Type":"ContainerDied","Data":"2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854"} Jul 01 18:22:01.971966 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.971822 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d" event={"ID":"20a33dae-136c-4a92-8a1c-e51e0ca7a640","Type":"ContainerDied","Data":"313e9fb2b23c4d25cf24d04e51d484b0938aaab51995a8defb0d1cd37da1ae59"} Jul 01 18:22:01.971966 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.971850 2572 scope.go:117] "RemoveContainer" containerID="a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5" Jul 01 18:22:01.980823 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.980809 2572 scope.go:117] "RemoveContainer" containerID="2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854" Jul 01 18:22:01.988001 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.987985 2572 scope.go:117] "RemoveContainer" containerID="7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f" Jul 01 18:22:01.994475 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.994452 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d"] Jul 01 18:22:01.995717 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.995705 2572 scope.go:117] "RemoveContainer" containerID="a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5" Jul 01 18:22:01.995992 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:01.995973 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5\": container with ID starting with a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5 not found: ID does not exist" containerID="a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5" Jul 01 18:22:01.996035 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.995999 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5"} err="failed to get container status \"a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5\": rpc error: code = NotFound desc = could not find container \"a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5\": container with ID starting with a6dff829840453793a9326dd7d243fe16133e5827ec72040891c239994802ff5 not found: ID does not exist" Jul 01 18:22:01.996035 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.996014 2572 scope.go:117] "RemoveContainer" containerID="2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854" Jul 01 18:22:01.996257 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:01.996239 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854\": container with ID starting with 2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854 not found: ID does not exist" containerID="2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854" Jul 01 18:22:01.996302 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.996264 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854"} err="failed to get container status \"2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854\": rpc error: code = NotFound desc = could not find container \"2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854\": container with ID starting with 2f8ad4e0fd80c7ae4423459ec0b6b15e41538d0c1cac8ba4792a648c34cf3854 not found: ID does not exist" Jul 01 18:22:01.996302 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.996280 2572 scope.go:117] "RemoveContainer" containerID="7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f" Jul 01 18:22:01.996487 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:01.996470 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f\": container with ID starting with 7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f not found: ID does not exist" containerID="7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f" Jul 01 18:22:01.996526 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.996489 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f"} err="failed to get container status \"7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f\": rpc error: code = NotFound desc = could not find container \"7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f\": container with ID starting with 7ee685cfea00d38ae8af606828efffc8e089cc1350acaa62856374758069538f not found: ID does not exist" Jul 01 18:22:01.998809 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:01.998788 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-hcw7d"] Jul 01 18:22:02.474169 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.474124 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.22:8643/healthz\": dial tcp 10.133.0.22:8643: connect: connection refused" Jul 01 18:22:02.480446 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.480422 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 18:22:02.795194 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.795175 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:22:02.834188 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.834169 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:22:02.912596 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912568 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b070bf96-654c-4b52-95dd-df1b1be88642-kserve-provision-location\") pod \"b070bf96-654c-4b52-95dd-df1b1be88642\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " Jul 01 18:22:02.912735 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912629 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-cabundle-cert\") pod \"58761e7f-5592-4b8d-a9c8-876f55a83277\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " Jul 01 18:22:02.912735 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912656 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sglr\" (UniqueName: \"kubernetes.io/projected/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-api-access-2sglr\") pod \"58761e7f-5592-4b8d-a9c8-876f55a83277\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " Jul 01 18:22:02.912735 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912695 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld87k\" (UniqueName: \"kubernetes.io/projected/b070bf96-654c-4b52-95dd-df1b1be88642-kube-api-access-ld87k\") pod \"b070bf96-654c-4b52-95dd-df1b1be88642\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " Jul 01 18:22:02.912735 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912727 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls\") pod \"58761e7f-5592-4b8d-a9c8-876f55a83277\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " Jul 01 18:22:02.912961 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912752 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/58761e7f-5592-4b8d-a9c8-876f55a83277-kserve-provision-location\") pod \"58761e7f-5592-4b8d-a9c8-876f55a83277\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " Jul 01 18:22:02.912961 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912853 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-kube-rbac-proxy-sar-config\") pod \"b070bf96-654c-4b52-95dd-df1b1be88642\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " Jul 01 18:22:02.912961 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912898 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-rbac-proxy-sar-config\") pod \"58761e7f-5592-4b8d-a9c8-876f55a83277\" (UID: \"58761e7f-5592-4b8d-a9c8-876f55a83277\") " Jul 01 18:22:02.912961 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912931 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b070bf96-654c-4b52-95dd-df1b1be88642-proxy-tls\") pod \"b070bf96-654c-4b52-95dd-df1b1be88642\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " Jul 01 18:22:02.912961 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912952 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-cabundle-cert\") pod \"b070bf96-654c-4b52-95dd-df1b1be88642\" (UID: \"b070bf96-654c-4b52-95dd-df1b1be88642\") " Jul 01 18:22:02.912961 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.912950 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b070bf96-654c-4b52-95dd-df1b1be88642-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b070bf96-654c-4b52-95dd-df1b1be88642" (UID: "b070bf96-654c-4b52-95dd-df1b1be88642"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:22:02.913246 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.913040 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "58761e7f-5592-4b8d-a9c8-876f55a83277" (UID: "58761e7f-5592-4b8d-a9c8-876f55a83277"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:22:02.913298 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.913284 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b070bf96-654c-4b52-95dd-df1b1be88642" (UID: "b070bf96-654c-4b52-95dd-df1b1be88642"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:22:02.913353 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.913289 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "58761e7f-5592-4b8d-a9c8-876f55a83277" (UID: "58761e7f-5592-4b8d-a9c8-876f55a83277"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:22:02.913353 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.913311 2572 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b070bf96-654c-4b52-95dd-df1b1be88642-kserve-provision-location\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:02.913353 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.913332 2572 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-cabundle-cert\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:02.913489 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.913378 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b070bf96-654c-4b52-95dd-df1b1be88642" (UID: "b070bf96-654c-4b52-95dd-df1b1be88642"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:22:02.913649 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.913627 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58761e7f-5592-4b8d-a9c8-876f55a83277-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "58761e7f-5592-4b8d-a9c8-876f55a83277" (UID: "58761e7f-5592-4b8d-a9c8-876f55a83277"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:22:02.915273 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.915248 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "58761e7f-5592-4b8d-a9c8-876f55a83277" (UID: "58761e7f-5592-4b8d-a9c8-876f55a83277"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:22:02.915273 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.915255 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-api-access-2sglr" (OuterVolumeSpecName: "kube-api-access-2sglr") pod "58761e7f-5592-4b8d-a9c8-876f55a83277" (UID: "58761e7f-5592-4b8d-a9c8-876f55a83277"). InnerVolumeSpecName "kube-api-access-2sglr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:22:02.915273 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.915267 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b070bf96-654c-4b52-95dd-df1b1be88642-kube-api-access-ld87k" (OuterVolumeSpecName: "kube-api-access-ld87k") pod "b070bf96-654c-4b52-95dd-df1b1be88642" (UID: "b070bf96-654c-4b52-95dd-df1b1be88642"). InnerVolumeSpecName "kube-api-access-ld87k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:22:02.915446 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.915262 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b070bf96-654c-4b52-95dd-df1b1be88642-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b070bf96-654c-4b52-95dd-df1b1be88642" (UID: "b070bf96-654c-4b52-95dd-df1b1be88642"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:22:02.977084 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.977056 2572 generic.go:356] "Generic (PLEG): container finished" podID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerID="a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2" exitCode=0 Jul 01 18:22:02.977179 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.977085 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" event={"ID":"58761e7f-5592-4b8d-a9c8-876f55a83277","Type":"ContainerDied","Data":"a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2"} Jul 01 18:22:02.977179 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.977123 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" Jul 01 18:22:02.977265 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.977185 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q" event={"ID":"58761e7f-5592-4b8d-a9c8-876f55a83277","Type":"ContainerDied","Data":"0918ecf1be23b10f502a15c8fd24947b1a39778611e3ffb2ec6de5d38f8cd6a5"} Jul 01 18:22:02.977265 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.977228 2572 scope.go:117] "RemoveContainer" containerID="b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a" Jul 01 18:22:02.979160 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.979136 2572 generic.go:356] "Generic (PLEG): container finished" podID="b070bf96-654c-4b52-95dd-df1b1be88642" containerID="8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b" exitCode=0 Jul 01 18:22:02.979258 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.979193 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" Jul 01 18:22:02.979258 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.979207 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" event={"ID":"b070bf96-654c-4b52-95dd-df1b1be88642","Type":"ContainerDied","Data":"8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b"} Jul 01 18:22:02.979366 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.979269 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf" event={"ID":"b070bf96-654c-4b52-95dd-df1b1be88642","Type":"ContainerDied","Data":"8fdf97b787d9d7a54f15382f974dd20b5d493aa38ab70384f4e2c8801c013bc5"} Jul 01 18:22:02.985735 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.985715 2572 scope.go:117] "RemoveContainer" containerID="a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2" Jul 01 18:22:02.993965 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:02.993949 2572 scope.go:117] "RemoveContainer" containerID="afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179" Jul 01 18:22:03.001195 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.001173 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q"] Jul 01 18:22:03.002432 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.002415 2572 scope.go:117] "RemoveContainer" containerID="b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a" Jul 01 18:22:03.002810 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:03.002701 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a\": container with ID starting with b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a not found: ID does not exist" containerID="b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a" Jul 01 18:22:03.002810 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.002732 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a"} err="failed to get container status \"b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a\": rpc error: code = NotFound desc = could not find container \"b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a\": container with ID starting with b4ec30677e477e31da249dc3be32957882bec49fe3a14f81a85d8ae96a86ef0a not found: ID does not exist" Jul 01 18:22:03.002810 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.002752 2572 scope.go:117] "RemoveContainer" containerID="a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2" Jul 01 18:22:03.003738 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:03.003702 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2\": container with ID starting with a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2 not found: ID does not exist" containerID="a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2" Jul 01 18:22:03.003859 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.003733 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2"} err="failed to get container status \"a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2\": rpc error: code = NotFound desc = could not find container \"a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2\": container with ID starting with a91c4cca9dad1fec0cdd9dc5cdab4a58f3eca1892a28728077eb48dc4148e9e2 not found: ID does not exist" Jul 01 18:22:03.003859 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.003752 2572 scope.go:117] "RemoveContainer" containerID="afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179" Jul 01 18:22:03.004037 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:03.004020 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179\": container with ID starting with afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179 not found: ID does not exist" containerID="afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179" Jul 01 18:22:03.004090 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.004040 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179"} err="failed to get container status \"afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179\": rpc error: code = NotFound desc = could not find container \"afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179\": container with ID starting with afc1e0235285cc85d8f624cebddef7bd8a2d0442a0d9f12bc87aae0b5b378179 not found: ID does not exist" Jul 01 18:22:03.004090 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.004053 2572 scope.go:117] "RemoveContainer" containerID="917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2" Jul 01 18:22:03.004542 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.004524 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-7c964dbd44-9zh7q"] Jul 01 18:22:03.010995 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.010978 2572 scope.go:117] "RemoveContainer" containerID="8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b" Jul 01 18:22:03.013709 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.013679 2572 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:03.013709 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.013701 2572 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:03.013709 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.013711 2572 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b070bf96-654c-4b52-95dd-df1b1be88642-proxy-tls\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:03.013896 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.013721 2572 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b070bf96-654c-4b52-95dd-df1b1be88642-cabundle-cert\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:03.013896 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.013730 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sglr\" (UniqueName: \"kubernetes.io/projected/58761e7f-5592-4b8d-a9c8-876f55a83277-kube-api-access-2sglr\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:03.013896 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.013738 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld87k\" (UniqueName: \"kubernetes.io/projected/b070bf96-654c-4b52-95dd-df1b1be88642-kube-api-access-ld87k\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:03.013896 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.013746 2572 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/58761e7f-5592-4b8d-a9c8-876f55a83277-proxy-tls\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:03.013896 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.013754 2572 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/58761e7f-5592-4b8d-a9c8-876f55a83277-kserve-provision-location\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:22:03.015449 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.015430 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf"] Jul 01 18:22:03.018246 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.018227 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-84b568d556-vxgvf"] Jul 01 18:22:03.018673 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.018651 2572 scope.go:117] "RemoveContainer" containerID="06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34" Jul 01 18:22:03.025315 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.025299 2572 scope.go:117] "RemoveContainer" containerID="917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2" Jul 01 18:22:03.025531 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:03.025515 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2\": container with ID starting with 917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2 not found: ID does not exist" containerID="917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2" Jul 01 18:22:03.025576 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.025537 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2"} err="failed to get container status \"917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2\": rpc error: code = NotFound desc = could not find container \"917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2\": container with ID starting with 917055d976331e124739ade6ac6d2011db01e8bc756da76609f9e2e23f70f5a2 not found: ID does not exist" Jul 01 18:22:03.025576 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.025557 2572 scope.go:117] "RemoveContainer" containerID="8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b" Jul 01 18:22:03.025822 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:03.025797 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b\": container with ID starting with 8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b not found: ID does not exist" containerID="8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b" Jul 01 18:22:03.025911 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.025822 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b"} err="failed to get container status \"8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b\": rpc error: code = NotFound desc = could not find container \"8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b\": container with ID starting with 8c10c1bcb054efd88c400df2a6fba3f2801664dddf877aba558662f77d02b41b not found: ID does not exist" Jul 01 18:22:03.025911 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.025837 2572 scope.go:117] "RemoveContainer" containerID="06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34" Jul 01 18:22:03.026079 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:22:03.026063 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34\": container with ID starting with 06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34 not found: ID does not exist" containerID="06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34" Jul 01 18:22:03.026117 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.026084 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34"} err="failed to get container status \"06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34\": rpc error: code = NotFound desc = could not find container \"06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34\": container with ID starting with 06972df3cff2086bbfdf6bfbcfaf0b0331735b499ce5e9e2810641541a4c1f34 not found: ID does not exist" Jul 01 18:22:03.360827 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.360737 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" path="/var/lib/kubelet/pods/20a33dae-136c-4a92-8a1c-e51e0ca7a640/volumes" Jul 01 18:22:03.361232 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.361218 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" path="/var/lib/kubelet/pods/58761e7f-5592-4b8d-a9c8-876f55a83277/volumes" Jul 01 18:22:03.361890 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:22:03.361869 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" path="/var/lib/kubelet/pods/b070bf96-654c-4b52-95dd-df1b1be88642/volumes" Jul 01 18:58:58.120755 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.120667 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7ccmx/must-gather-t88x2"] Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121147 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121164 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121176 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121184 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121203 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="storage-initializer" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121209 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="storage-initializer" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121216 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121221 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121231 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="storage-initializer" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121236 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="storage-initializer" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121242 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="storage-initializer" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121247 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="storage-initializer" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121255 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121261 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121274 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121279 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121286 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121291 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121345 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121353 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121359 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b070bf96-654c-4b52-95dd-df1b1be88642" containerName="kserve-container" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121366 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kube-rbac-proxy" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121374 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a33dae-136c-4a92-8a1c-e51e0ca7a640" containerName="kserve-container" Jul 01 18:58:58.123347 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.121381 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="58761e7f-5592-4b8d-a9c8-876f55a83277" containerName="kserve-container" Jul 01 18:58:58.124261 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.124243 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:58:58.126648 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.126628 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7ccmx\"/\"openshift-service-ca.crt\"" Jul 01 18:58:58.127566 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.127549 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7ccmx\"/\"kube-root-ca.crt\"" Jul 01 18:58:58.127566 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.127555 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7ccmx\"/\"default-dockercfg-6wp6c\"" Jul 01 18:58:58.132340 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.132314 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7ccmx/must-gather-t88x2"] Jul 01 18:58:58.195124 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.195093 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9414b6c0-9b71-452e-9d29-4d3637037aa3-must-gather-output\") pod \"must-gather-t88x2\" (UID: \"9414b6c0-9b71-452e-9d29-4d3637037aa3\") " pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:58:58.195282 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.195141 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrdwf\" (UniqueName: \"kubernetes.io/projected/9414b6c0-9b71-452e-9d29-4d3637037aa3-kube-api-access-wrdwf\") pod \"must-gather-t88x2\" (UID: \"9414b6c0-9b71-452e-9d29-4d3637037aa3\") " pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:58:58.296010 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.295972 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9414b6c0-9b71-452e-9d29-4d3637037aa3-must-gather-output\") pod \"must-gather-t88x2\" (UID: \"9414b6c0-9b71-452e-9d29-4d3637037aa3\") " pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:58:58.296192 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.296034 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrdwf\" (UniqueName: \"kubernetes.io/projected/9414b6c0-9b71-452e-9d29-4d3637037aa3-kube-api-access-wrdwf\") pod \"must-gather-t88x2\" (UID: \"9414b6c0-9b71-452e-9d29-4d3637037aa3\") " pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:58:58.296343 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.296322 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9414b6c0-9b71-452e-9d29-4d3637037aa3-must-gather-output\") pod \"must-gather-t88x2\" (UID: \"9414b6c0-9b71-452e-9d29-4d3637037aa3\") " pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:58:58.304627 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.304605 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrdwf\" (UniqueName: \"kubernetes.io/projected/9414b6c0-9b71-452e-9d29-4d3637037aa3-kube-api-access-wrdwf\") pod \"must-gather-t88x2\" (UID: \"9414b6c0-9b71-452e-9d29-4d3637037aa3\") " pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:58:58.442542 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.442452 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:58:58.570562 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.570536 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7ccmx/must-gather-t88x2"] Jul 01 18:58:58.572981 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:58:58.572953 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9414b6c0_9b71_452e_9d29_4d3637037aa3.slice/crio-7d41169b8e691e9f821a1c2ea80336f11848e2f652e8f8cb774737b50c819edb WatchSource:0}: Error finding container 7d41169b8e691e9f821a1c2ea80336f11848e2f652e8f8cb774737b50c819edb: Status 404 returned error can't find the container with id 7d41169b8e691e9f821a1c2ea80336f11848e2f652e8f8cb774737b50c819edb Jul 01 18:58:58.574910 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.574892 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 18:58:58.638096 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:58:58.638059 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ccmx/must-gather-t88x2" event={"ID":"9414b6c0-9b71-452e-9d29-4d3637037aa3","Type":"ContainerStarted","Data":"7d41169b8e691e9f821a1c2ea80336f11848e2f652e8f8cb774737b50c819edb"} Jul 01 18:59:05.673594 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:05.673555 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ccmx/must-gather-t88x2" event={"ID":"9414b6c0-9b71-452e-9d29-4d3637037aa3","Type":"ContainerStarted","Data":"92eb69e68598576813d804197819422995f3c07f1a67ca3f3caf908c01d80e46"} Jul 01 18:59:05.674080 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:05.673604 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ccmx/must-gather-t88x2" event={"ID":"9414b6c0-9b71-452e-9d29-4d3637037aa3","Type":"ContainerStarted","Data":"6bc15b64ba4bc9c5fdf824749efbc0c5a4e6a42eb848ee740b17f373903b115b"} Jul 01 18:59:05.690194 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:05.690149 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7ccmx/must-gather-t88x2" podStartSLOduration=1.690785414 podStartE2EDuration="7.690134487s" totalImagesPullingTime="5.999349073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:59:05.689310737 +0000 UTC m=+3154.830494110" watchObservedRunningTime="2026-07-01 18:59:05.690134487 +0000 UTC m=+3154.831317859" Jul 01 18:59:28.779037 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:28.778995 2572 generic.go:356] "Generic (PLEG): container finished" podID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerID="6bc15b64ba4bc9c5fdf824749efbc0c5a4e6a42eb848ee740b17f373903b115b" exitCode=0 Jul 01 18:59:28.779453 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:28.779062 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7ccmx/must-gather-t88x2" event={"ID":"9414b6c0-9b71-452e-9d29-4d3637037aa3","Type":"ContainerDied","Data":"6bc15b64ba4bc9c5fdf824749efbc0c5a4e6a42eb848ee740b17f373903b115b"} Jul 01 18:59:28.779453 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:28.779359 2572 scope.go:117] "RemoveContainer" containerID="6bc15b64ba4bc9c5fdf824749efbc0c5a4e6a42eb848ee740b17f373903b115b" Jul 01 18:59:29.078403 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:29.078321 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-7ccmx_must-gather-t88x2_9414b6c0-9b71-452e-9d29-4d3637037aa3/gather/0.log" Jul 01 18:59:32.476516 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:32.476477 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5jn49_46476d5f-12a3-4783-b432-854f32042355/global-pull-secret-syncer/0.log" Jul 01 18:59:32.710658 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:32.710629 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vhlpx_a9418d21-5deb-43b9-aa86-476f01d902e5/konnectivity-agent/0.log" Jul 01 18:59:32.815447 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:32.815342 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-152.ec2.internal_a4e6868fe585dc3b6f2f3608909f610d/haproxy/0.log" Jul 01 18:59:34.672508 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.672471 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-7ccmx/must-gather-t88x2"] Jul 01 18:59:34.673028 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.672663 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-7ccmx/must-gather-t88x2" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerName="copy" containerID="cri-o://92eb69e68598576813d804197819422995f3c07f1a67ca3f3caf908c01d80e46" gracePeriod=2 Jul 01 18:59:34.678788 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.678746 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-7ccmx/must-gather-t88x2"] Jul 01 18:59:34.680165 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:59:34.680131 2572 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9414b6c0-9b71-452e-9d29-4d3637037aa3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-7ccmx\"/\"must-gather-t88x2\": pods \"must-gather-t88x2\" not found" pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:59:34.803563 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.803539 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-7ccmx_must-gather-t88x2_9414b6c0-9b71-452e-9d29-4d3637037aa3/copy/0.log" Jul 01 18:59:34.803986 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.803955 2572 generic.go:356] "Generic (PLEG): container finished" podID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerID="92eb69e68598576813d804197819422995f3c07f1a67ca3f3caf908c01d80e46" exitCode=143 Jul 01 18:59:34.894900 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.894879 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-7ccmx_must-gather-t88x2_9414b6c0-9b71-452e-9d29-4d3637037aa3/copy/0.log" Jul 01 18:59:34.896513 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.896496 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:59:34.898832 ip-10-0-136-152 kubenswrapper[2572]: E0701 18:59:34.898755 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-t88x2\" is forbidden: User \"system:node:ip-10-0-136-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-7ccmx\": no relationship found between node 'ip-10-0-136-152.ec2.internal' and this object" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:59:34.998248 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.998189 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrdwf\" (UniqueName: \"kubernetes.io/projected/9414b6c0-9b71-452e-9d29-4d3637037aa3-kube-api-access-wrdwf\") pod \"9414b6c0-9b71-452e-9d29-4d3637037aa3\" (UID: \"9414b6c0-9b71-452e-9d29-4d3637037aa3\") " Jul 01 18:59:34.998248 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.998230 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9414b6c0-9b71-452e-9d29-4d3637037aa3-must-gather-output\") pod \"9414b6c0-9b71-452e-9d29-4d3637037aa3\" (UID: \"9414b6c0-9b71-452e-9d29-4d3637037aa3\") " Jul 01 18:59:34.999766 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:34.999743 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9414b6c0-9b71-452e-9d29-4d3637037aa3-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "9414b6c0-9b71-452e-9d29-4d3637037aa3" (UID: "9414b6c0-9b71-452e-9d29-4d3637037aa3"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:59:35.000315 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:35.000295 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9414b6c0-9b71-452e-9d29-4d3637037aa3-kube-api-access-wrdwf" (OuterVolumeSpecName: "kube-api-access-wrdwf") pod "9414b6c0-9b71-452e-9d29-4d3637037aa3" (UID: "9414b6c0-9b71-452e-9d29-4d3637037aa3"). InnerVolumeSpecName "kube-api-access-wrdwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:59:35.099367 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:35.099335 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrdwf\" (UniqueName: \"kubernetes.io/projected/9414b6c0-9b71-452e-9d29-4d3637037aa3-kube-api-access-wrdwf\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:59:35.099367 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:35.099364 2572 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9414b6c0-9b71-452e-9d29-4d3637037aa3-must-gather-output\") on node \"ip-10-0-136-152.ec2.internal\" DevicePath \"\"" Jul 01 18:59:35.360922 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:35.360842 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" path="/var/lib/kubelet/pods/9414b6c0-9b71-452e-9d29-4d3637037aa3/volumes" Jul 01 18:59:35.809322 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:35.809295 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-7ccmx_must-gather-t88x2_9414b6c0-9b71-452e-9d29-4d3637037aa3/copy/0.log" Jul 01 18:59:35.809757 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:35.809653 2572 scope.go:117] "RemoveContainer" containerID="92eb69e68598576813d804197819422995f3c07f1a67ca3f3caf908c01d80e46" Jul 01 18:59:35.809757 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:35.809694 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7ccmx/must-gather-t88x2" Jul 01 18:59:35.818957 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:35.818942 2572 scope.go:117] "RemoveContainer" containerID="6bc15b64ba4bc9c5fdf824749efbc0c5a4e6a42eb848ee740b17f373903b115b" Jul 01 18:59:36.129113 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.129088 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-t7phw_933d818a-0783-4070-9394-957b53f50751/cluster-monitoring-operator/0.log" Jul 01 18:59:36.158577 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.158555 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-x77tx_97b391cd-4f83-4497-82de-54e288f77664/kube-state-metrics/0.log" Jul 01 18:59:36.192592 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.192558 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-x77tx_97b391cd-4f83-4497-82de-54e288f77664/kube-rbac-proxy-main/0.log" Jul 01 18:59:36.224530 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.224506 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-x77tx_97b391cd-4f83-4497-82de-54e288f77664/kube-rbac-proxy-self/0.log" Jul 01 18:59:36.516899 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.516808 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qzrn_1f17c45b-6fb0-4003-9054-3fb5c3dc056e/node-exporter/0.log" Jul 01 18:59:36.543007 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.542982 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qzrn_1f17c45b-6fb0-4003-9054-3fb5c3dc056e/kube-rbac-proxy/0.log" Jul 01 18:59:36.571189 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.571166 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qzrn_1f17c45b-6fb0-4003-9054-3fb5c3dc056e/init-textfile/0.log" Jul 01 18:59:36.718254 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.718219 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-rkv7k_ad051676-d1e8-428f-853e-ee2b2082d4a9/kube-rbac-proxy-main/0.log" Jul 01 18:59:36.743792 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.743744 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-rkv7k_ad051676-d1e8-428f-853e-ee2b2082d4a9/kube-rbac-proxy-self/0.log" Jul 01 18:59:36.768558 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.768480 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-rkv7k_ad051676-d1e8-428f-853e-ee2b2082d4a9/openshift-state-metrics/0.log" Jul 01 18:59:36.823658 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.823630 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c184770e-e529-4cf1-aa14-b45f6c41bf64/prometheus/0.log" Jul 01 18:59:36.844657 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.844631 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c184770e-e529-4cf1-aa14-b45f6c41bf64/config-reloader/0.log" Jul 01 18:59:36.873663 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.873637 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c184770e-e529-4cf1-aa14-b45f6c41bf64/thanos-sidecar/0.log" Jul 01 18:59:36.899796 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.899761 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c184770e-e529-4cf1-aa14-b45f6c41bf64/kube-rbac-proxy-web/0.log" Jul 01 18:59:36.925488 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.925462 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c184770e-e529-4cf1-aa14-b45f6c41bf64/kube-rbac-proxy/0.log" Jul 01 18:59:36.949034 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.948986 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c184770e-e529-4cf1-aa14-b45f6c41bf64/kube-rbac-proxy-thanos/0.log" Jul 01 18:59:36.973764 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:36.973736 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c184770e-e529-4cf1-aa14-b45f6c41bf64/init-config-reloader/0.log" Jul 01 18:59:37.003460 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.003435 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-kvkfc_3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7/prometheus-operator/0.log" Jul 01 18:59:37.021642 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.021576 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-kvkfc_3f0f7491-df74-4f2a-b6d3-fe3b6ffd77a7/kube-rbac-proxy/0.log" Jul 01 18:59:37.049473 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.049445 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-xrgk4_2d208d24-280f-4f55-98dd-b3e38415885c/prometheus-operator-admission-webhook/0.log" Jul 01 18:59:37.171974 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.171942 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/thanos-query/0.log" Jul 01 18:59:37.193368 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.193341 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/kube-rbac-proxy-web/0.log" Jul 01 18:59:37.217089 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.217061 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/kube-rbac-proxy/0.log" Jul 01 18:59:37.242239 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.242213 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/prom-label-proxy/0.log" Jul 01 18:59:37.263664 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.263645 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/kube-rbac-proxy-rules/0.log" Jul 01 18:59:37.285589 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:37.285504 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-969f7c748-r26br_3eb0d90d-d86a-4182-941f-0986044e6265/kube-rbac-proxy-metrics/0.log" Jul 01 18:59:39.879764 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.879730 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2"] Jul 01 18:59:39.880214 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.880065 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerName="gather" Jul 01 18:59:39.880214 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.880079 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerName="gather" Jul 01 18:59:39.880214 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.880098 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerName="copy" Jul 01 18:59:39.880214 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.880103 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerName="copy" Jul 01 18:59:39.880214 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.880180 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerName="copy" Jul 01 18:59:39.880214 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.880188 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9414b6c0-9b71-452e-9d29-4d3637037aa3" containerName="gather" Jul 01 18:59:39.884268 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.884252 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:39.887501 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.887479 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lpb44\"/\"openshift-service-ca.crt\"" Jul 01 18:59:39.887767 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.887744 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lpb44\"/\"default-dockercfg-gczn5\"" Jul 01 18:59:39.888538 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.888522 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lpb44\"/\"kube-root-ca.crt\"" Jul 01 18:59:39.893358 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.893339 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2"] Jul 01 18:59:39.936085 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.936063 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-proc\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:39.936180 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.936089 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-sys\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:39.936180 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.936133 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-podres\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:39.936180 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.936167 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-lib-modules\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:39.936282 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:39.936188 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hczf\" (UniqueName: \"kubernetes.io/projected/af8973ff-e35f-4912-b8cb-88b69620665d-kube-api-access-4hczf\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037293 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037270 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-podres\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037411 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-lib-modules\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037411 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037325 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hczf\" (UniqueName: \"kubernetes.io/projected/af8973ff-e35f-4912-b8cb-88b69620665d-kube-api-access-4hczf\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037411 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037344 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-proc\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037411 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037367 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-sys\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037561 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037432 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-lib-modules\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037561 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037437 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-podres\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037561 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037433 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-proc\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.037561 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.037438 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af8973ff-e35f-4912-b8cb-88b69620665d-sys\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.045654 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.045627 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hczf\" (UniqueName: \"kubernetes.io/projected/af8973ff-e35f-4912-b8cb-88b69620665d-kube-api-access-4hczf\") pod \"perf-node-gather-daemonset-blvb2\" (UID: \"af8973ff-e35f-4912-b8cb-88b69620665d\") " pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.193950 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.193865 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.320103 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.319886 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2"] Jul 01 18:59:40.322187 ip-10-0-136-152 kubenswrapper[2572]: W0701 18:59:40.322160 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaf8973ff_e35f_4912_b8cb_88b69620665d.slice/crio-50a28a7f2bc04e7198a9e8be7dee4ebabb7ac7297b1ed24f62db2989b66b9fd5 WatchSource:0}: Error finding container 50a28a7f2bc04e7198a9e8be7dee4ebabb7ac7297b1ed24f62db2989b66b9fd5: Status 404 returned error can't find the container with id 50a28a7f2bc04e7198a9e8be7dee4ebabb7ac7297b1ed24f62db2989b66b9fd5 Jul 01 18:59:40.831357 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.831265 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" event={"ID":"af8973ff-e35f-4912-b8cb-88b69620665d","Type":"ContainerStarted","Data":"2dd416ad3c2d2d2fabc9e8c9e32bc5f2f51e4340f9db16e6dc38424016ff5cbf"} Jul 01 18:59:40.831357 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.831322 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" event={"ID":"af8973ff-e35f-4912-b8cb-88b69620665d","Type":"ContainerStarted","Data":"50a28a7f2bc04e7198a9e8be7dee4ebabb7ac7297b1ed24f62db2989b66b9fd5"} Jul 01 18:59:40.831584 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.831382 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:40.849049 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:40.849003 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" podStartSLOduration=1.848989102 podStartE2EDuration="1.848989102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:59:40.847628071 +0000 UTC m=+3189.988811457" watchObservedRunningTime="2026-07-01 18:59:40.848989102 +0000 UTC m=+3189.990172481" Jul 01 18:59:41.098934 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:41.098851 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wmv7g_2c83a996-6382-4768-ba63-7d3a5067953a/dns/0.log" Jul 01 18:59:41.120350 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:41.120323 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wmv7g_2c83a996-6382-4768-ba63-7d3a5067953a/kube-rbac-proxy/0.log" Jul 01 18:59:41.145395 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:41.145369 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f5fv7_8528571c-8df9-4351-b771-3d37effe01ca/dns-node-resolver/0.log" Jul 01 18:59:41.723526 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:41.723495 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lnrs5_9de04a82-ce0c-4dfa-b8e1-1db529bdfbab/node-ca/0.log" Jul 01 18:59:42.507145 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:42.507113 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86dc5bb4b9-fkb96_3c23be7d-3674-4403-b8ba-31854ca8de3c/router/0.log" Jul 01 18:59:42.907592 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:42.907558 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zbkc9_c6c1a534-badc-41f6-95a8-2ebcba6394fc/serve-healthcheck-canary/0.log" Jul 01 18:59:43.255285 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:43.255192 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwwtp_44927caa-061c-4efe-a8e3-4ff490799539/kube-rbac-proxy/0.log" Jul 01 18:59:43.276028 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:43.276000 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwwtp_44927caa-061c-4efe-a8e3-4ff490799539/exporter/0.log" Jul 01 18:59:43.297674 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:43.297648 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwwtp_44927caa-061c-4efe-a8e3-4ff490799539/extractor/0.log" Jul 01 18:59:45.586790 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:45.586744 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-bdft9_2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0/s3-init/0.log" Jul 01 18:59:45.606365 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:45.606343 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-bdft9_2c9c7398-72d3-4dc2-966b-0c1c4a3db4b0/download-hf-model/0.log" Jul 01 18:59:46.844843 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:46.844815 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lpb44/perf-node-gather-daemonset-blvb2" Jul 01 18:59:49.705592 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:49.705516 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-hc7x6_bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e/migrator/0.log" Jul 01 18:59:49.726014 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:49.725992 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-hc7x6_bc6ac817-f50c-4bd3-aa74-a28ec6d34f4e/graceful-termination/0.log" Jul 01 18:59:51.144574 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.144552 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sftwb_5f9ea0bd-275c-4941-abea-106b2c01ab01/kube-multus-additional-cni-plugins/0.log" Jul 01 18:59:51.165542 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.165520 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sftwb_5f9ea0bd-275c-4941-abea-106b2c01ab01/egress-router-binary-copy/0.log" Jul 01 18:59:51.186788 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.186744 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sftwb_5f9ea0bd-275c-4941-abea-106b2c01ab01/cni-plugins/0.log" Jul 01 18:59:51.206630 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.206605 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sftwb_5f9ea0bd-275c-4941-abea-106b2c01ab01/bond-cni-plugin/0.log" Jul 01 18:59:51.228481 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.228464 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sftwb_5f9ea0bd-275c-4941-abea-106b2c01ab01/routeoverride-cni/0.log" Jul 01 18:59:51.249330 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.249307 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sftwb_5f9ea0bd-275c-4941-abea-106b2c01ab01/whereabouts-cni-bincopy/0.log" Jul 01 18:59:51.270222 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.270200 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sftwb_5f9ea0bd-275c-4941-abea-106b2c01ab01/whereabouts-cni/0.log" Jul 01 18:59:51.467681 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.467653 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-m2r5j_c1b2d4a9-d472-4ac8-a33f-af4d676109ab/kube-multus/0.log" Jul 01 18:59:51.646280 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.646251 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dr9zg_d7e0bb01-e2e5-41b6-a552-ae7b6380eaec/network-metrics-daemon/0.log" Jul 01 18:59:51.666601 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:51.666577 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dr9zg_d7e0bb01-e2e5-41b6-a552-ae7b6380eaec/kube-rbac-proxy/0.log" Jul 01 18:59:52.910433 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:52.910394 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xmjfl_627519ad-b343-43ab-8f0f-9fc101b7cb91/ovn-controller/0.log" Jul 01 18:59:52.928605 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:52.928583 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xmjfl_627519ad-b343-43ab-8f0f-9fc101b7cb91/ovn-acl-logging/0.log" Jul 01 18:59:52.962492 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:52.962468 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xmjfl_627519ad-b343-43ab-8f0f-9fc101b7cb91/kube-rbac-proxy-node/0.log" Jul 01 18:59:52.987142 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:52.987114 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xmjfl_627519ad-b343-43ab-8f0f-9fc101b7cb91/kube-rbac-proxy-ovn-metrics/0.log" Jul 01 18:59:53.006762 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:53.006742 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xmjfl_627519ad-b343-43ab-8f0f-9fc101b7cb91/northd/0.log" Jul 01 18:59:53.029094 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:53.029072 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xmjfl_627519ad-b343-43ab-8f0f-9fc101b7cb91/nbdb/0.log" Jul 01 18:59:53.053100 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:53.053082 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xmjfl_627519ad-b343-43ab-8f0f-9fc101b7cb91/sbdb/0.log" Jul 01 18:59:53.111748 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:53.111722 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xmjfl_627519ad-b343-43ab-8f0f-9fc101b7cb91/ovnkube-controller/0.log" Jul 01 18:59:54.138686 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:54.138649 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-sddsx_9395f543-f981-48e3-8a19-d285b1bc933f/network-check-target-container/0.log" Jul 01 18:59:55.090902 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:55.090870 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-mrltk_250e8f25-8966-4f14-83ab-1e5b42aeb73a/iptables-alerter/0.log" Jul 01 18:59:55.894965 ip-10-0-136-152 kubenswrapper[2572]: I0701 18:59:55.894935 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-pl4gm_4d53392f-dc5c-4868-8447-4a5f8c5f6f16/tuned/0.log"