Jul 07 12:48:36.755221 ip-10-0-131-17 systemd[1]: Starting Kubernetes Kubelet... Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.234271 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241843 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241859 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241863 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241866 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241870 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241873 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241875 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:48:37.273022 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241878 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241882 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241884 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241888 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241890 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241893 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241896 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241898 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241900 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241903 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241906 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241908 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241911 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241913 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241916 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241919 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241921 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241924 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241927 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241932 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:48:37.274696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241935 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241937 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241940 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241942 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241945 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241947 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241950 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241952 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241956 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241958 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241961 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241963 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241985 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241988 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241990 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241993 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241996 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.241998 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242002 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242005 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:48:37.275281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242007 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242012 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242015 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242018 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242021 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242025 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242027 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242029 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242032 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242035 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242037 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242040 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242043 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242045 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242048 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242050 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242053 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242056 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242058 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:48:37.275816 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242061 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242063 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242066 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242069 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242071 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242074 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242077 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242080 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242083 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242085 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242087 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242090 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242093 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242097 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242101 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242104 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242106 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242110 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242112 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242115 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:48:37.441940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242118 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:48:37.317490 ip-10-0-131-17 systemd[1]: Started Kubernetes Kubelet. Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242120 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242123 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242126 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242128 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242131 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242133 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242136 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242139 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242142 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242144 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242147 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242149 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242835 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242841 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242844 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242848 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242850 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242853 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:48:37.455117 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242856 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242858 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242861 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242863 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242866 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242869 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242871 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242874 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242876 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242878 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242881 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242884 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242887 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242889 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242892 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242894 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242897 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242900 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242902 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242904 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:48:37.456003 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242909 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242912 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242915 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242918 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242920 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242923 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242926 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242929 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242932 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242935 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242938 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242940 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242943 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242945 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242948 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242950 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242953 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242956 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242958 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242962 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:48:37.456894 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242964 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242984 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242987 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242989 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242992 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242995 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.242998 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243000 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243002 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243005 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243007 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243010 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243012 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243015 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243019 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243023 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243026 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243028 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243031 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243034 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:48:37.457713 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243036 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243040 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243043 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243046 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243049 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243051 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243054 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243056 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243059 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243061 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243064 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243067 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243070 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243072 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243074 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243077 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243079 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243083 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243090 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:48:37.458320 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243093 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243096 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243099 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243102 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243104 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243107 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243109 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243112 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243115 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243117 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243120 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243122 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243125 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243127 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243513 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243518 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243523 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243526 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243530 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243533 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:48:37.458925 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243536 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243539 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243542 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243545 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243548 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243550 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243554 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243556 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243559 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243561 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243564 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243569 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243572 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243575 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243577 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243580 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243583 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243585 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243588 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243590 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:48:37.459527 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243593 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243595 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243599 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243603 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243605 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243608 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243618 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243621 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243624 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243627 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243629 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243632 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243634 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243637 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243639 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243642 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243644 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243648 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243651 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243654 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:48:37.460159 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243657 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243661 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243664 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243668 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243671 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243674 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243676 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243679 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243681 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243684 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243686 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243689 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243692 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243694 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243697 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243699 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243702 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243706 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243708 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243711 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:48:37.460789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243713 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243716 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243718 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243721 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243725 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243727 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243730 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243733 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243736 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243738 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243740 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243743 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243745 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243748 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243750 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243754 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243757 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243759 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243762 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:48:37.461435 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243766 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243769 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243771 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243774 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243777 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243779 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243782 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243785 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243787 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243790 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243794 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243796 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243799 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.243801 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243865 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243872 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243878 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243883 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243887 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243890 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243893 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 12:48:37.461990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243897 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243900 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243903 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243907 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243910 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243913 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243916 2568 flags.go:64] FLAG: --cgroup-root="" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243920 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243923 2568 flags.go:64] FLAG: --client-ca-file="" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243927 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243930 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243934 2568 flags.go:64] FLAG: --cluster-domain="" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243937 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243940 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243943 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243946 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243950 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243952 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243955 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243959 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243961 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243965 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243982 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243986 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 12:48:37.462627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243989 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243993 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243996 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.243999 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244002 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244004 2568 flags.go:64] FLAG: --enable-server="true" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244007 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244012 2568 flags.go:64] FLAG: --event-burst="100" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244015 2568 flags.go:64] FLAG: --event-qps="50" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244018 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244021 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244024 2568 flags.go:64] FLAG: --eviction-hard="" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244027 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244030 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244033 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244038 2568 flags.go:64] FLAG: --eviction-soft="" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244041 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244043 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244047 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244050 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244053 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244055 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244058 2568 flags.go:64] FLAG: --feature-gates="" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244062 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244065 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 12:48:37.463247 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244068 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244071 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244074 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244076 2568 flags.go:64] FLAG: --help="false" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244082 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-131-17.ec2.internal" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244085 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244088 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244091 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244094 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244097 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244100 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244103 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244106 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244108 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244111 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244114 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244117 2568 flags.go:64] FLAG: --kube-reserved="" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244120 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244123 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244126 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244128 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244131 2568 flags.go:64] FLAG: --lock-file="" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244135 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244138 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 12:48:37.463936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244141 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244146 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244150 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244152 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244155 2568 flags.go:64] FLAG: --logging-format="text" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244158 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244161 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244173 2568 flags.go:64] FLAG: --manifest-url="" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244176 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244181 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244184 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244187 2568 flags.go:64] FLAG: --max-pods="110" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244192 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244195 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244198 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244201 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244204 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244207 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244210 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244217 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244221 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244223 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244226 2568 flags.go:64] FLAG: --pod-cidr="" Jul 07 12:48:37.464631 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244229 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244234 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244237 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244240 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244243 2568 flags.go:64] FLAG: --port="10250" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244246 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244248 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-040bcb27415c8764d" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244252 2568 flags.go:64] FLAG: --qos-reserved="" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244256 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244259 2568 flags.go:64] FLAG: --register-node="true" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244262 2568 flags.go:64] FLAG: --register-with-taints="" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244265 2568 flags.go:64] FLAG: --registry-burst="10" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244270 2568 flags.go:64] FLAG: --registry-qps="5" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244272 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244275 2568 flags.go:64] FLAG: --reserved-memory="" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244279 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244282 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244285 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244287 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244290 2568 flags.go:64] FLAG: --runonce="false" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244293 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244298 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244301 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244304 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244307 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244310 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 12:48:37.465299 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244312 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244315 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244318 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244321 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244324 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244327 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244330 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244332 2568 flags.go:64] FLAG: --system-cgroups="" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244335 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244340 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244343 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244346 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244350 2568 flags.go:64] FLAG: --tls-min-version="" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244353 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244356 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244359 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244362 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244365 2568 flags.go:64] FLAG: --v="2" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244368 2568 flags.go:64] FLAG: --version="false" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244373 2568 flags.go:64] FLAG: --vmodule="" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244376 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.244380 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244495 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244498 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:48:37.466281 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244501 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244504 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244506 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244510 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244514 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244518 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244521 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244524 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244527 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244529 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244532 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244534 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244537 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244539 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244542 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244544 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244547 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244549 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244552 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244555 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:48:37.466940 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244557 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244560 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244564 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244566 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244569 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244572 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244574 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244577 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244580 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244582 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244585 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244587 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244590 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244592 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244595 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244598 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244601 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244603 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244606 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244609 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:48:38.817421 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244611 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244614 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244616 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244619 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244621 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244624 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244627 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244629 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244632 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244634 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244636 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244639 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244642 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244644 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244648 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244651 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244653 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244656 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244658 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:48:38.818599 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244661 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244663 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244667 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244670 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244672 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244674 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244677 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244681 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244685 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244688 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244691 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244694 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244696 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244699 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244701 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244703 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244706 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244708 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244711 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:48:38.819253 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244713 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244716 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244719 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244723 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244726 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244729 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244731 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244734 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244737 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244740 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244743 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244745 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244748 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244751 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244753 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244756 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244759 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244761 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:48:38.819845 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.244763 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.245434 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.253226 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.253246 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.253269 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.253277 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253351 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253358 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253362 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253366 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253371 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253375 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253379 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253383 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253387 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:48:38.820352 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253391 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253395 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253398 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253402 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253406 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253411 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253414 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253418 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253422 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253426 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253430 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253434 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253437 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253441 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253445 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253448 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253451 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253455 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253460 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253464 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:48:38.820811 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253469 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253472 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253476 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253480 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253484 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253487 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253491 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253495 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253499 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253502 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253506 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253510 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253514 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253518 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253522 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253526 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253530 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253533 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253537 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253541 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:48:38.821357 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253544 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253548 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253552 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253556 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253561 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253565 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253568 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253572 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253577 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253580 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253584 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253591 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253595 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253599 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253603 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253607 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253611 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253617 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253622 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253625 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:48:38.821931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253630 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253633 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253637 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253641 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253645 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253648 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253652 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253656 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253659 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253665 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253670 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253674 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253679 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253683 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253687 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253691 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253697 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253701 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253705 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:48:38.822563 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253709 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253713 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253717 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253721 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253727 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253731 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253735 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253739 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253743 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253747 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253752 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.253760 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253915 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253921 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:48:38.823384 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253926 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253930 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253934 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253939 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253943 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253946 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253950 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253954 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253958 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253961 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253981 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253985 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253989 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253993 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.253997 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254001 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254006 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254010 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254014 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254019 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:48:38.823789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254022 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254026 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254032 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254036 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254040 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254044 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254049 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254053 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254056 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254060 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254064 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254067 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254071 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254075 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254079 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254083 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254087 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254090 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254095 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:48:38.824397 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254099 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254103 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254106 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254110 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254113 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254117 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254121 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254124 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254128 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254134 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254138 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254144 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254147 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254151 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254156 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254160 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254164 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254168 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254172 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254175 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:48:38.824987 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254180 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254184 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254187 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254191 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254195 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254198 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254202 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254206 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254209 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254213 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254216 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254221 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254224 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254229 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254232 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254236 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254239 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254243 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254247 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:48:38.825597 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254250 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254254 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254261 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254265 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254269 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254273 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254277 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254280 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254287 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254290 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254296 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254301 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254304 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254309 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254314 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254317 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254321 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254325 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:48:38.826095 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:37.254328 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.254335 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.254460 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.258065 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.259518 2568 server.go:1019] "Starting client certificate rotation" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.259614 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.260290 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.262546 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.270944 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.283264 2568 log.go:25] "Validated CRI v1 runtime API" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.285338 2568 log.go:25] "Validated CRI v1 image API" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.286096 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 12:48:38.826684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.291673 2568 fs.go:135] Filesystem UUIDs: map[0fc26d98-13c5-4778-8603-0c229566013e:/dev/nvme0n1p4 68d7d98c-7f04-4418-bde7-099fc7935e05:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 07 12:48:38.827064 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.291690 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 12:48:38.827182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.296584 2568 manager.go:216] Machine: {Timestamp:2026-07-07 12:48:37.295236468 +0000 UTC m=+0.398229198 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106586 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a23b0c0f95926ffe8d285b1b98383 SystemUUID:ec2a23b0-c0f9-5926-ffe8-d285b1b98383 BootID:7693f0b8-3eac-4cf4-8d32-ab50dbbdb6ba Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:58:d9:c1:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:58:d9:c1:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:5b:7f:fb:31:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 12:48:38.827182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.296715 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 12:48:38.827182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.296843 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 12:48:38.827182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.300775 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 12:48:38.827182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.300797 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 12:48:38.827182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.302389 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 12:48:38.827182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.302399 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 12:48:38.827182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.302407 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.302420 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.302861 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.303936 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307397 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307500 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307529 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307546 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307565 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307575 2568 kubelet.go:398] "Adding apiserver pod source" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307585 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307856 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.307870 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.310334 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.312392 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.312419 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313717 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313738 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313744 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313750 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313758 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313763 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313770 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313775 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313782 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313787 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313801 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313806 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313828 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.313834 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.316725 2568 server.go:1268] "Started kubelet" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.316808 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.316881 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.316946 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.318249 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.321106 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-17.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.324153 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.324480 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.324781 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.327022 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 12:48:38.827457 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.327093 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.327218 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.327435 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.327701 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.327703 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.327721 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.327818 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.327825 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.328005 2568 factory.go:55] Registering systemd factory Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.328042 2568 factory.go:223] Registration of the systemd container factory successfully Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.328242 2568 factory.go:153] Registering CRI-O factory Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.328257 2568 factory.go:223] Registration of the crio container factory successfully Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.328305 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.328330 2568 factory.go:103] Registering Raw factory Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.328344 2568 manager.go:1196] Started watching for new ooms in manager Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.328638 2568 manager.go:349] Starting recovery of all containers Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.327155 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-17.ec2.internal.18c002c998f6a0e8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-17.ec2.internal,UID:ip-10-0-131-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-17.ec2.internal,},FirstTimestamp:2026-07-07 12:48:37.316698344 +0000 UTC m=+0.419691077,LastTimestamp:2026-07-07 12:48:37.316698344 +0000 UTC m=+0.419691077,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-17.ec2.internal,}" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.329732 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.329825 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.340655 2568 manager.go:354] Recovery completed Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.342041 2568 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.345519 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.346897 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vrgzx" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.347236 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.347259 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.347269 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.347762 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.347770 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.347773 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.347784 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 07 12:48:38.828524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.348462 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.348487 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.348510 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.349066 2568 policy_none.go:49] "None policy: Start" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.349080 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.349090 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.349675 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-17.ec2.internal.18c002c99ac8bfad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-17.ec2.internal,UID:ip-10-0-131-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-17.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-17.ec2.internal,},FirstTimestamp:2026-07-07 12:48:37.347245997 +0000 UTC m=+0.450238727,LastTimestamp:2026-07-07 12:48:37.347245997 +0000 UTC m=+0.450238727,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-17.ec2.internal,}" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.350098 2568 policy_none.go:47] "Start" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.364041 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vrgzx" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.368566 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-17.ec2.internal.18c002c99ac905b3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-17.ec2.internal,UID:ip-10-0-131-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-17.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-17.ec2.internal,},FirstTimestamp:2026-07-07 12:48:37.347263923 +0000 UTC m=+0.450256653,LastTimestamp:2026-07-07 12:48:37.347263923 +0000 UTC m=+0.450256653,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-17.ec2.internal,}" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.382501 2568 manager.go:337] "Starting Device Plugin manager" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.382529 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.382538 2568 server.go:87] "Starting device plugin registration server" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.382724 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.382737 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.382830 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.382922 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.382935 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.383390 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.383443 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.477429 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.478613 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.478641 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.478662 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.478700 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.481036 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.483140 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:48:38.829733 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.483884 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.483910 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.483924 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.483943 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.496604 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.496620 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-17.ec2.internal\": node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.534117 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.579177 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal"] Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.579229 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.580824 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.580847 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.580858 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582070 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582176 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582203 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582677 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582678 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582714 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582725 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582699 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.582788 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.583865 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.583887 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.590531 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.590553 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:37.590562 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.599533 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-17.ec2.internal\" not found" node="ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:37.605420 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-17.ec2.internal\" not found" node="ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.059552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.748370 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73b41ec98bd430c3a24e32263117d9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal\" (UID: \"c73b41ec98bd430c3a24e32263117d9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.748529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73b41ec98bd430c3a24e32263117d9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal\" (UID: \"c73b41ec98bd430c3a24e32263117d9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.748546 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e331315f490aea5c3012f6891135f171-config\") pod \"kube-apiserver-proxy-ip-10-0-131-17.ec2.internal\" (UID: \"e331315f490aea5c3012f6891135f171\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:38.753383 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.849260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73b41ec98bd430c3a24e32263117d9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal\" (UID: \"c73b41ec98bd430c3a24e32263117d9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.849282 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e331315f490aea5c3012f6891135f171-config\") pod \"kube-apiserver-proxy-ip-10-0-131-17.ec2.internal\" (UID: \"e331315f490aea5c3012f6891135f171\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.849299 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73b41ec98bd430c3a24e32263117d9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal\" (UID: \"c73b41ec98bd430c3a24e32263117d9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.849323 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73b41ec98bd430c3a24e32263117d9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal\" (UID: \"c73b41ec98bd430c3a24e32263117d9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.849364 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73b41ec98bd430c3a24e32263117d9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal\" (UID: \"c73b41ec98bd430c3a24e32263117d9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:38.849364 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e331315f490aea5c3012f6891135f171-config\") pod \"kube-apiserver-proxy-ip-10-0-131-17.ec2.internal\" (UID: \"e331315f490aea5c3012f6891135f171\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:38.857627 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:38.957681 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.020916 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.027011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.062828 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:39.058169 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.299419 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:39.158687 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.299419 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:39.259207 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.494549 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:39.359689 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.494549 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.377832 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 12:48:39.494549 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.377945 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 12:48:39.494549 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:39.459866 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.494549 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.486254 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 12:43:38 +0000 UTC" deadline="2028-03-12 04:17:18.186593808 +0000 UTC" Jul 07 12:48:39.494549 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.486286 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14727h28m38.70031114s" Jul 07 12:48:39.658756 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.506231 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:48:39.658756 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:39.560510 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.658756 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.596672 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:48:39.658756 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.601830 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:48:39.658756 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.602754 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:48:39.658756 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.602782 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:48:39.658756 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.602792 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:39.660575 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-17.ec2.internal\" not found" Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:39.660569 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73b41ec98bd430c3a24e32263117d9d.slice/crio-6af70c36e4e3ce17944cda7510ed35c4c544722580bb0bf44b441b43ce32b80f WatchSource:0}: Error finding container 6af70c36e4e3ce17944cda7510ed35c4c544722580bb0bf44b441b43ce32b80f: Status 404 returned error can't find the container with id 6af70c36e4e3ce17944cda7510ed35c4c544722580bb0bf44b441b43ce32b80f Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:39.660982 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode331315f490aea5c3012f6891135f171.slice/crio-1f594bd35f67653639f2e0039c6b10764295efb6121437e43ddffb56d9e23283 WatchSource:0}: Error finding container 1f594bd35f67653639f2e0039c6b10764295efb6121437e43ddffb56d9e23283: Status 404 returned error can't find the container with id 1f594bd35f67653639f2e0039c6b10764295efb6121437e43ddffb56d9e23283 Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.665846 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.700042 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.746337 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.762686 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.764009 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" Jul 07 12:48:39.849596 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:39.775445 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.427134 2568 apiserver.go:52] "Watching apiserver" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.435262 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.435718 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-qrwz8","kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal","openshift-cluster-node-tuning-operator/tuned-dxd2p","openshift-dns/node-resolver-9n6g2","openshift-multus/multus-additional-cni-plugins-tcbbf","openshift-multus/network-metrics-daemon-kzrz6","openshift-network-operator/iptables-alerter-l8j8h","openshift-ovn-kubernetes/ovnkube-node-4hjrd","kube-system/global-pull-secret-syncer-jqxc9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk","openshift-image-registry/node-ca-42l45","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal","openshift-multus/multus-fqg8q","openshift-network-diagnostics/network-check-target-mv8d9"] Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.437369 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.438982 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.441804 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2hrbr\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.442106 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.442359 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.442668 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.444040 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.445657 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.446005 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.446586 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.449701 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4vbsx\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.450564 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.451278 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-78cqx\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.451445 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.451469 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.452898 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.453154 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.453339 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.453505 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.453659 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.453863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.454006 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:40.483634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.455774 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.456525 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.456633 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rqcxf\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.456925 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.457007 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.457067 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.457254 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.457262 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b56zz\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.457393 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.458363 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.458809 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.458968 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c7xfq\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.459064 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.460358 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.461980 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.461992 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sxmrw\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.461981 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.462226 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.463029 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.463386 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.463647 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-99lk5\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.463738 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.463772 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.465688 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.465691 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.466035 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.466286 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.466383 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5dwf5\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.467322 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.467525 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 12:48:40.484905 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.477144 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.488802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.488833 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c097b54b-f582-4f51-a99c-308303239efe-iptables-alerter-script\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.488857 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c097b54b-f582-4f51-a99c-308303239efe-host-slash\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.488888 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-os-release\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.488910 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-hostroot\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.488947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c45wz\" (UniqueName: \"kubernetes.io/projected/3fa538d7-30b0-42e6-bb5f-2233438fd583-kube-api-access-c45wz\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.488986 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2be6b55-d608-438f-9bcd-2d8f9849cb16-hosts-file\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-system-cni-dir\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489073 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-os-release\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysctl-d\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.489147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489143 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysctl-conf\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489230 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-sys\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-lib-modules\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489302 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fa538d7-30b0-42e6-bb5f-2233438fd583-cni-binary-copy\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-socket-dir-parent\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489378 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-k8s-cni-cncf-io\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489401 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkb4q\" (UniqueName: \"kubernetes.io/projected/c097b54b-f582-4f51-a99c-308303239efe-kube-api-access-dkb4q\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489422 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-netns\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-cni-bin\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-daemon-config\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-cnibin\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-tuned\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489584 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:40.492583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489606 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2be6b55-d608-438f-9bcd-2d8f9849cb16-tmp-dir\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489629 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghkmd\" (UniqueName: \"kubernetes.io/projected/b2be6b55-d608-438f-9bcd-2d8f9849cb16-kube-api-access-ghkmd\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489655 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-kubernetes\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489716 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-host\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489737 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-cni-multus\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-cni-binary-copy\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489781 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90229356-0deb-4f7d-bc00-f8ba96b684ff-tmp\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77djt\" (UniqueName: \"kubernetes.io/projected/d719beb6-22cd-4784-a0bb-463c9a856741-kube-api-access-77djt\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489827 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/275c0b51-3ae0-4606-8672-f047fa541200-kubelet-config\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489866 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-conf-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489890 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489912 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktqx6\" (UniqueName: \"kubernetes.io/projected/0915ebda-b0cc-492b-91fc-1f92045f243f-kube-api-access-ktqx6\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.493605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489959 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-modprobe-d\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.489991 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-var-lib-kubelet\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/275c0b51-3ae0-4606-8672-f047fa541200-dbus\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-system-cni-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-cni-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490130 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-cnibin\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490158 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysconfig\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490180 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-systemd\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490204 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-run\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59qkd\" (UniqueName: \"kubernetes.io/projected/90229356-0deb-4f7d-bc00-f8ba96b684ff-kube-api-access-59qkd\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490277 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-kubelet\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490316 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-multus-certs\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.498786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.490361 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-etc-kubernetes\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.504416 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.502613 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cbtbc" Jul 07 12:48:40.567767 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.515331 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cbtbc" Jul 07 12:48:40.567767 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.548550 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 12:48:40.583747 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.582448 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.590959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-netns\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.590991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-daemon-config\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591018 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591042 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-slash\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591072 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-netns\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591077 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-var-lib-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-etc-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591191 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovn-node-metrics-cert\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2be6b55-d608-438f-9bcd-2d8f9849cb16-tmp-dir\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghkmd\" (UniqueName: \"kubernetes.io/projected/b2be6b55-d608-438f-9bcd-2d8f9849cb16-kube-api-access-ghkmd\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591397 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-host\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-cni-binary-copy\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.660690 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591446 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591451 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-host\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591486 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2be6b55-d608-438f-9bcd-2d8f9849cb16-tmp-dir\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-ovn\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-node-log\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591602 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-daemon-config\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591686 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktqx6\" (UniqueName: \"kubernetes.io/projected/0915ebda-b0cc-492b-91fc-1f92045f243f-kube-api-access-ktqx6\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591710 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d45af9fd-3eb6-4979-87e3-7e084b78aad5-agent-certs\") pod \"konnectivity-agent-qrwz8\" (UID: \"d45af9fd-3eb6-4979-87e3-7e084b78aad5\") " pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591733 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/275c0b51-3ae0-4606-8672-f047fa541200-dbus\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-system-cni-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591792 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.781950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-systemd\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591871 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovnkube-script-lib\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591881 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-system-cni-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591895 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-run\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591924 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-run-netns\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591944 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovnkube-config\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2be6b55-d608-438f-9bcd-2d8f9849cb16-hosts-file\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592001 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/275c0b51-3ae0-4606-8672-f047fa541200-dbus\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592015 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-run\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592024 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-cni-binary-copy\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.591992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592066 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-878l2\" (UniqueName: \"kubernetes.io/projected/621128a0-56bf-42dc-9566-2ddd2cf5bc58-kube-api-access-878l2\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592067 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2be6b55-d608-438f-9bcd-2d8f9849cb16-hosts-file\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592092 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-log-socket\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.782997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592128 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-cni-bin\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqgc7\" (UniqueName: \"kubernetes.io/projected/1542a9c1-a15f-4413-acc5-fff6edbadd71-kube-api-access-fqgc7\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592169 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysctl-conf\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592170 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-lib-modules\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592208 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-k8s-cni-cncf-io\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592233 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-cni-netd\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592242 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-env-overrides\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-lib-modules\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592293 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-k8s-cni-cncf-io\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592308 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-cnibin\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592311 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysctl-conf\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592324 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-kubelet\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.783869 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592364 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-tuned\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592395 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-cnibin\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592452 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/621128a0-56bf-42dc-9566-2ddd2cf5bc58-host\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-kubernetes\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592506 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-kubernetes\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592510 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-cni-multus\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.592526 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592543 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-socket-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592538 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-cni-multus\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592574 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90229356-0deb-4f7d-bc00-f8ba96b684ff-tmp\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592612 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77djt\" (UniqueName: \"kubernetes.io/projected/d719beb6-22cd-4784-a0bb-463c9a856741-kube-api-access-77djt\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592644 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.592655 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs podName:d719beb6-22cd-4784-a0bb-463c9a856741 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:41.092636 +0000 UTC m=+3.077484468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs") pod "network-metrics-daemon-kzrz6" (UID: "d719beb6-22cd-4784-a0bb-463c9a856741") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:40.784669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/275c0b51-3ae0-4606-8672-f047fa541200-kubelet-config\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592724 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-conf-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592754 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-modprobe-d\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592789 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-var-lib-kubelet\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-cni-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592810 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-conf-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592838 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/275c0b51-3ae0-4606-8672-f047fa541200-kubelet-config\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592871 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-var-lib-kubelet\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592883 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-modprobe-d\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592898 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-cni-dir\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-cnibin\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592937 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-cnibin\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-multus-certs\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592971 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-registration-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.785256 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592982 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-run-multus-certs\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.592997 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysconfig\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-systemd\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59qkd\" (UniqueName: \"kubernetes.io/projected/90229356-0deb-4f7d-bc00-f8ba96b684ff-kube-api-access-59qkd\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593057 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysconfig\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593074 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-kubelet\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593100 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-etc-kubernetes\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593148 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d45af9fd-3eb6-4979-87e3-7e084b78aad5-konnectivity-ca\") pod \"konnectivity-agent-qrwz8\" (UID: \"d45af9fd-3eb6-4979-87e3-7e084b78aad5\") " pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593149 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-kubelet\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593102 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-systemd\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593179 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-etc-kubernetes\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-device-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593219 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0915ebda-b0cc-492b-91fc-1f92045f243f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593231 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-sys-fs\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.785973 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c097b54b-f582-4f51-a99c-308303239efe-iptables-alerter-script\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c097b54b-f582-4f51-a99c-308303239efe-host-slash\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593307 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-os-release\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593333 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-hostroot\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45wz\" (UniqueName: \"kubernetes.io/projected/3fa538d7-30b0-42e6-bb5f-2233438fd583-kube-api-access-c45wz\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593354 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c097b54b-f582-4f51-a99c-308303239efe-host-slash\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-system-cni-dir\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-os-release\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593396 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-hostroot\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593407 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-os-release\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593416 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57qgn\" (UniqueName: \"kubernetes.io/projected/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-kube-api-access-57qgn\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593435 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-system-cni-dir\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysctl-d\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593448 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0915ebda-b0cc-492b-91fc-1f92045f243f-os-release\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593461 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-sys\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.786567 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fa538d7-30b0-42e6-bb5f-2233438fd583-cni-binary-copy\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593511 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-sysctl-d\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593528 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-socket-dir-parent\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90229356-0deb-4f7d-bc00-f8ba96b684ff-sys\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593560 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-cni-bin\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593581 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/621128a0-56bf-42dc-9566-2ddd2cf5bc58-serviceca\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.593592 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-systemd-units\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.593622 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret podName:275c0b51-3ae0-4606-8672-f047fa541200 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:41.093611868 +0000 UTC m=+3.078460318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret") pod "global-pull-secret-syncer-jqxc9" (UID: "275c0b51-3ae0-4606-8672-f047fa541200") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593641 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkb4q\" (UniqueName: \"kubernetes.io/projected/c097b54b-f582-4f51-a99c-308303239efe-kube-api-access-dkb4q\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593658 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-host-var-lib-cni-bin\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593692 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3fa538d7-30b0-42e6-bb5f-2233438fd583-multus-socket-dir-parent\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.593727 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c097b54b-f582-4f51-a99c-308303239efe-iptables-alerter-script\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.787306 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.594088 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fa538d7-30b0-42e6-bb5f-2233438fd583-cni-binary-copy\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.596437 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90229356-0deb-4f7d-bc00-f8ba96b684ff-tmp\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.596488 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90229356-0deb-4f7d-bc00-f8ba96b684ff-etc-tuned\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.607410 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" event={"ID":"e331315f490aea5c3012f6891135f171","Type":"ContainerStarted","Data":"1f594bd35f67653639f2e0039c6b10764295efb6121437e43ddffb56d9e23283"} Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.607473 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" event={"ID":"c73b41ec98bd430c3a24e32263117d9d","Type":"ContainerStarted","Data":"6af70c36e4e3ce17944cda7510ed35c4c544722580bb0bf44b441b43ce32b80f"} Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.609253 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77djt\" (UniqueName: \"kubernetes.io/projected/d719beb6-22cd-4784-a0bb-463c9a856741-kube-api-access-77djt\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.609680 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkb4q\" (UniqueName: \"kubernetes.io/projected/c097b54b-f582-4f51-a99c-308303239efe-kube-api-access-dkb4q\") pod \"iptables-alerter-l8j8h\" (UID: \"c097b54b-f582-4f51-a99c-308303239efe\") " pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.610407 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghkmd\" (UniqueName: \"kubernetes.io/projected/b2be6b55-d608-438f-9bcd-2d8f9849cb16-kube-api-access-ghkmd\") pod \"node-resolver-9n6g2\" (UID: \"b2be6b55-d608-438f-9bcd-2d8f9849cb16\") " pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.610962 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59qkd\" (UniqueName: \"kubernetes.io/projected/90229356-0deb-4f7d-bc00-f8ba96b684ff-kube-api-access-59qkd\") pod \"tuned-dxd2p\" (UID: \"90229356-0deb-4f7d-bc00-f8ba96b684ff\") " pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.611890 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45wz\" (UniqueName: \"kubernetes.io/projected/3fa538d7-30b0-42e6-bb5f-2233438fd583-kube-api-access-c45wz\") pod \"multus-fqg8q\" (UID: \"3fa538d7-30b0-42e6-bb5f-2233438fd583\") " pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.613337 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktqx6\" (UniqueName: \"kubernetes.io/projected/0915ebda-b0cc-492b-91fc-1f92045f243f-kube-api-access-ktqx6\") pod \"multus-additional-cni-plugins-tcbbf\" (UID: \"0915ebda-b0cc-492b-91fc-1f92045f243f\") " pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.694798 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/621128a0-56bf-42dc-9566-2ddd2cf5bc58-serviceca\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.694839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-systemd-units\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.694868 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.694896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-slash\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.787979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.694922 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-var-lib-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.694947 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-etc-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.694972 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovn-node-metrics-cert\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695027 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695069 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695094 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-ovn\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-node-log\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695169 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695197 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695226 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d45af9fd-3eb6-4979-87e3-7e084b78aad5-agent-certs\") pod \"konnectivity-agent-qrwz8\" (UID: \"d45af9fd-3eb6-4979-87e3-7e084b78aad5\") " pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-systemd\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695283 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovnkube-script-lib\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-run-netns\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.788561 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695324 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/621128a0-56bf-42dc-9566-2ddd2cf5bc58-serviceca\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695340 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovnkube-config\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878l2\" (UniqueName: \"kubernetes.io/projected/621128a0-56bf-42dc-9566-2ddd2cf5bc58-kube-api-access-878l2\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695398 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-log-socket\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695426 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-cni-bin\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695453 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqgc7\" (UniqueName: \"kubernetes.io/projected/1542a9c1-a15f-4413-acc5-fff6edbadd71-kube-api-access-fqgc7\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695461 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695474 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-var-lib-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695483 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695489 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-cni-netd\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695510 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-etc-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-node-log\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695453 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-ovn\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695541 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-cni-netd\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695583 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-cni-bin\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695592 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-env-overrides\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-kubelet\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/621128a0-56bf-42dc-9566-2ddd2cf5bc58-host\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695714 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-run-netns\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695722 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-systemd-units\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695760 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-systemd\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695817 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-run-openvswitch\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695830 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-slash\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695631 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-log-socket\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695924 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.695974 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/621128a0-56bf-42dc-9566-2ddd2cf5bc58-host\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696008 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1542a9c1-a15f-4413-acc5-fff6edbadd71-host-kubelet\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-socket-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.789727 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696171 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-socket-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696179 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-registration-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696209 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d45af9fd-3eb6-4979-87e3-7e084b78aad5-konnectivity-ca\") pod \"konnectivity-agent-qrwz8\" (UID: \"d45af9fd-3eb6-4979-87e3-7e084b78aad5\") " pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696220 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovnkube-config\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-device-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696246 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovnkube-script-lib\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696254 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-device-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-sys-fs\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696290 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-registration-dir\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57qgn\" (UniqueName: \"kubernetes.io/projected/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-kube-api-access-57qgn\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696291 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1542a9c1-a15f-4413-acc5-fff6edbadd71-env-overrides\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696346 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-sys-fs\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.696717 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d45af9fd-3eb6-4979-87e3-7e084b78aad5-konnectivity-ca\") pod \"konnectivity-agent-qrwz8\" (UID: \"d45af9fd-3eb6-4979-87e3-7e084b78aad5\") " pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.698340 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d45af9fd-3eb6-4979-87e3-7e084b78aad5-agent-certs\") pod \"konnectivity-agent-qrwz8\" (UID: \"d45af9fd-3eb6-4979-87e3-7e084b78aad5\") " pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.699999 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1542a9c1-a15f-4413-acc5-fff6edbadd71-ovn-node-metrics-cert\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.790383 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.719795 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.719815 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.719825 2568 projected.go:196] Error preparing data for projected volume kube-api-access-h5qrp for pod openshift-network-diagnostics/network-check-target-mv8d9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:40.719874 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp podName:5e34485f-5517-4209-ba1b-44ff7ebae506 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:41.219861517 +0000 UTC m=+3.204709967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h5qrp" (UniqueName: "kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp") pod "network-check-target-mv8d9" (UID: "5e34485f-5517-4209-ba1b-44ff7ebae506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.722658 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-878l2\" (UniqueName: \"kubernetes.io/projected/621128a0-56bf-42dc-9566-2ddd2cf5bc58-kube-api-access-878l2\") pod \"node-ca-42l45\" (UID: \"621128a0-56bf-42dc-9566-2ddd2cf5bc58\") " pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.723981 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqgc7\" (UniqueName: \"kubernetes.io/projected/1542a9c1-a15f-4413-acc5-fff6edbadd71-kube-api-access-fqgc7\") pod \"ovnkube-node-4hjrd\" (UID: \"1542a9c1-a15f-4413-acc5-fff6edbadd71\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.724692 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57qgn\" (UniqueName: \"kubernetes.io/projected/4a7301a6-3201-4be0-a95d-ac453a6ff1f5-kube-api-access-57qgn\") pod \"aws-ebs-csi-driver-node-rp5lk\" (UID: \"4a7301a6-3201-4be0-a95d-ac453a6ff1f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.763842 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l8j8h" Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.780946 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fqg8q" Jul 07 12:48:40.791054 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.788070 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" Jul 07 12:48:40.794034 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.794008 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" Jul 07 12:48:40.802170 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.802151 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9n6g2" Jul 07 12:48:40.815812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.815794 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:48:40.823537 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.823485 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" Jul 07 12:48:40.830163 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.830145 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-42l45" Jul 07 12:48:40.837655 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:40.837610 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:48:41.098332 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:41.098261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:41.098332 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:41.098314 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:41.098499 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:41.098425 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:41.098499 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:41.098481 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret podName:275c0b51-3ae0-4606-8672-f047fa541200 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:42.098467558 +0000 UTC m=+4.083316013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret") pod "global-pull-secret-syncer-jqxc9" (UID: "275c0b51-3ae0-4606-8672-f047fa541200") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:41.098587 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:41.098427 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:41.098587 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:41.098570 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs podName:d719beb6-22cd-4784-a0bb-463c9a856741 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:42.098551973 +0000 UTC m=+4.083400424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs") pod "network-metrics-daemon-kzrz6" (UID: "d719beb6-22cd-4784-a0bb-463c9a856741") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:41.300841 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:41.300808 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:41.300999 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:41.300932 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:48:41.300999 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:41.300951 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:48:41.300999 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:41.300961 2568 projected.go:196] Error preparing data for projected volume kube-api-access-h5qrp for pod openshift-network-diagnostics/network-check-target-mv8d9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:41.301145 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:41.301010 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp podName:5e34485f-5517-4209-ba1b-44ff7ebae506 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:42.300994377 +0000 UTC m=+4.285842848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5qrp" (UniqueName: "kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp") pod "network-check-target-mv8d9" (UID: "5e34485f-5517-4209-ba1b-44ff7ebae506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:41.518586 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:41.518490 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 12:43:40 +0000 UTC" deadline="2028-03-01 20:31:00.409811313 +0000 UTC" Jul 07 12:48:41.518586 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:41.518533 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14479h42m18.891281582s" Jul 07 12:48:41.712264 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.712240 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc097b54b_f582_4f51_a99c_308303239efe.slice/crio-b9dbaaf666f9b2d0cd0b7abb9e1402617c8a4a73beba76d697e15b2d255829e7 WatchSource:0}: Error finding container b9dbaaf666f9b2d0cd0b7abb9e1402617c8a4a73beba76d697e15b2d255829e7: Status 404 returned error can't find the container with id b9dbaaf666f9b2d0cd0b7abb9e1402617c8a4a73beba76d697e15b2d255829e7 Jul 07 12:48:41.729960 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.729883 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0915ebda_b0cc_492b_91fc_1f92045f243f.slice/crio-33630f0cf044f79bec7946efeb33c72475cd36b8ca5025a41ebd28765d8f3e4b WatchSource:0}: Error finding container 33630f0cf044f79bec7946efeb33c72475cd36b8ca5025a41ebd28765d8f3e4b: Status 404 returned error can't find the container with id 33630f0cf044f79bec7946efeb33c72475cd36b8ca5025a41ebd28765d8f3e4b Jul 07 12:48:41.731200 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.731175 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7301a6_3201_4be0_a95d_ac453a6ff1f5.slice/crio-9688ff05b7bf4f906743a4b500b9e34059988f673fbdbf0c921fe3cc7e21b18d WatchSource:0}: Error finding container 9688ff05b7bf4f906743a4b500b9e34059988f673fbdbf0c921fe3cc7e21b18d: Status 404 returned error can't find the container with id 9688ff05b7bf4f906743a4b500b9e34059988f673fbdbf0c921fe3cc7e21b18d Jul 07 12:48:41.731931 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.731908 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2be6b55_d608_438f_9bcd_2d8f9849cb16.slice/crio-1ee54dbb488da6cb0213095fb060a8263cb76234c1b28681c1c2e02ba2cd2145 WatchSource:0}: Error finding container 1ee54dbb488da6cb0213095fb060a8263cb76234c1b28681c1c2e02ba2cd2145: Status 404 returned error can't find the container with id 1ee54dbb488da6cb0213095fb060a8263cb76234c1b28681c1c2e02ba2cd2145 Jul 07 12:48:41.732583 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.732561 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621128a0_56bf_42dc_9566_2ddd2cf5bc58.slice/crio-37cfb38152d42a84ccfdc219c37445ec6f4d4fb1a00e0d5e01858fee956fc257 WatchSource:0}: Error finding container 37cfb38152d42a84ccfdc219c37445ec6f4d4fb1a00e0d5e01858fee956fc257: Status 404 returned error can't find the container with id 37cfb38152d42a84ccfdc219c37445ec6f4d4fb1a00e0d5e01858fee956fc257 Jul 07 12:48:41.734242 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.734221 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45af9fd_3eb6_4979_87e3_7e084b78aad5.slice/crio-5267fae44c76729c40990d472b80c6bd6341229fdbaaab733f4e6ded4e41c40f WatchSource:0}: Error finding container 5267fae44c76729c40990d472b80c6bd6341229fdbaaab733f4e6ded4e41c40f: Status 404 returned error can't find the container with id 5267fae44c76729c40990d472b80c6bd6341229fdbaaab733f4e6ded4e41c40f Jul 07 12:48:41.735013 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.734982 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa538d7_30b0_42e6_bb5f_2233438fd583.slice/crio-6c0006932187fa68200dfa0ca4cd6b6ff1d56acbcdc63013d46bb777ae4e4140 WatchSource:0}: Error finding container 6c0006932187fa68200dfa0ca4cd6b6ff1d56acbcdc63013d46bb777ae4e4140: Status 404 returned error can't find the container with id 6c0006932187fa68200dfa0ca4cd6b6ff1d56acbcdc63013d46bb777ae4e4140 Jul 07 12:48:41.735994 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.735971 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90229356_0deb_4f7d_bc00_f8ba96b684ff.slice/crio-61c7492a3567b5cd4e8a9bd761e50f60c954ecb11c73acc433d7fecbd85d981c WatchSource:0}: Error finding container 61c7492a3567b5cd4e8a9bd761e50f60c954ecb11c73acc433d7fecbd85d981c: Status 404 returned error can't find the container with id 61c7492a3567b5cd4e8a9bd761e50f60c954ecb11c73acc433d7fecbd85d981c Jul 07 12:48:41.737068 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:48:41.737042 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1542a9c1_a15f_4413_acc5_fff6edbadd71.slice/crio-7a511cdf7747eb7dfa9ecbfac37f2737d52efe15f00dbcdb052abdc329115442 WatchSource:0}: Error finding container 7a511cdf7747eb7dfa9ecbfac37f2737d52efe15f00dbcdb052abdc329115442: Status 404 returned error can't find the container with id 7a511cdf7747eb7dfa9ecbfac37f2737d52efe15f00dbcdb052abdc329115442 Jul 07 12:48:42.107133 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.106876 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:42.107242 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.107019 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:42.107242 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.107158 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:42.107325 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.107243 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret podName:275c0b51-3ae0-4606-8672-f047fa541200 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:44.1071894 +0000 UTC m=+6.092037862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret") pod "global-pull-secret-syncer-jqxc9" (UID: "275c0b51-3ae0-4606-8672-f047fa541200") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:42.107325 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.107274 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:42.107325 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.107325 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs podName:d719beb6-22cd-4784-a0bb-463c9a856741 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:44.107314088 +0000 UTC m=+6.092162538 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs") pod "network-metrics-daemon-kzrz6" (UID: "d719beb6-22cd-4784-a0bb-463c9a856741") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:42.319133 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.309022 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:42.322463 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.319549 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:48:42.322463 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.319574 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:48:42.322463 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.319594 2568 projected.go:196] Error preparing data for projected volume kube-api-access-h5qrp for pod openshift-network-diagnostics/network-check-target-mv8d9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:42.322463 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.319663 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp podName:5e34485f-5517-4209-ba1b-44ff7ebae506 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:44.319636556 +0000 UTC m=+6.304485029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5qrp" (UniqueName: "kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp") pod "network-check-target-mv8d9" (UID: "5e34485f-5517-4209-ba1b-44ff7ebae506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:42.519193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.519019 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 12:43:40 +0000 UTC" deadline="2028-03-24 17:35:46.766484384 +0000 UTC" Jul 07 12:48:42.519193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.519085 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15028h47m4.247403741s" Jul 07 12:48:42.600340 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.600311 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:42.600497 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.600447 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:42.600557 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.600544 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:42.600650 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.600612 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:42.600705 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.600671 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:42.600763 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:42.600732 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:42.615855 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.615195 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" event={"ID":"e331315f490aea5c3012f6891135f171","Type":"ContainerStarted","Data":"d1a1b056f5861dc6e03f887c532470ae06682089acec10b80b84f987db8a53ce"} Jul 07 12:48:42.626277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.626216 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fqg8q" event={"ID":"3fa538d7-30b0-42e6-bb5f-2233438fd583","Type":"ContainerStarted","Data":"6c0006932187fa68200dfa0ca4cd6b6ff1d56acbcdc63013d46bb777ae4e4140"} Jul 07 12:48:42.634180 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.634151 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrwz8" event={"ID":"d45af9fd-3eb6-4979-87e3-7e084b78aad5","Type":"ContainerStarted","Data":"5267fae44c76729c40990d472b80c6bd6341229fdbaaab733f4e6ded4e41c40f"} Jul 07 12:48:42.646514 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.646490 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-42l45" event={"ID":"621128a0-56bf-42dc-9566-2ddd2cf5bc58","Type":"ContainerStarted","Data":"37cfb38152d42a84ccfdc219c37445ec6f4d4fb1a00e0d5e01858fee956fc257"} Jul 07 12:48:42.649957 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.649933 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" event={"ID":"4a7301a6-3201-4be0-a95d-ac453a6ff1f5","Type":"ContainerStarted","Data":"9688ff05b7bf4f906743a4b500b9e34059988f673fbdbf0c921fe3cc7e21b18d"} Jul 07 12:48:42.657495 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.657468 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9n6g2" event={"ID":"b2be6b55-d608-438f-9bcd-2d8f9849cb16","Type":"ContainerStarted","Data":"1ee54dbb488da6cb0213095fb060a8263cb76234c1b28681c1c2e02ba2cd2145"} Jul 07 12:48:42.659246 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.659223 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" event={"ID":"0915ebda-b0cc-492b-91fc-1f92045f243f","Type":"ContainerStarted","Data":"33630f0cf044f79bec7946efeb33c72475cd36b8ca5025a41ebd28765d8f3e4b"} Jul 07 12:48:42.666754 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.666731 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l8j8h" event={"ID":"c097b54b-f582-4f51-a99c-308303239efe","Type":"ContainerStarted","Data":"b9dbaaf666f9b2d0cd0b7abb9e1402617c8a4a73beba76d697e15b2d255829e7"} Jul 07 12:48:42.671967 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.671939 2568 generic.go:356] "Generic (PLEG): container finished" podID="c73b41ec98bd430c3a24e32263117d9d" containerID="a969f1d4d196bd5333acd66b46df481a920c3c39fe102e2715dfc2219a1d65bd" exitCode=0 Jul 07 12:48:42.672242 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.672221 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" event={"ID":"c73b41ec98bd430c3a24e32263117d9d","Type":"ContainerDied","Data":"a969f1d4d196bd5333acd66b46df481a920c3c39fe102e2715dfc2219a1d65bd"} Jul 07 12:48:42.677035 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.677010 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"7a511cdf7747eb7dfa9ecbfac37f2737d52efe15f00dbcdb052abdc329115442"} Jul 07 12:48:42.679765 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.679743 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" event={"ID":"90229356-0deb-4f7d-bc00-f8ba96b684ff","Type":"ContainerStarted","Data":"61c7492a3567b5cd4e8a9bd761e50f60c954ecb11c73acc433d7fecbd85d981c"} Jul 07 12:48:42.700351 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:42.700293 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-17.ec2.internal" podStartSLOduration=3.700275112 podStartE2EDuration="3.700275112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:42.643047913 +0000 UTC m=+4.627896384" watchObservedRunningTime="2026-07-07 12:48:42.700275112 +0000 UTC m=+4.685123586" Jul 07 12:48:43.699251 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:43.699201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" event={"ID":"c73b41ec98bd430c3a24e32263117d9d","Type":"ContainerStarted","Data":"a397554820f714b8e7d2baf4e8decb005003ee59849b72e43c4b663e064c35e4"} Jul 07 12:48:43.720071 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:43.719976 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-17.ec2.internal" podStartSLOduration=4.719957127 podStartE2EDuration="4.719957127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:43.719105823 +0000 UTC m=+5.703954296" watchObservedRunningTime="2026-07-07 12:48:43.719957127 +0000 UTC m=+5.704805602" Jul 07 12:48:44.124508 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:44.124429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:44.124508 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:44.124496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:44.124721 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.124647 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:44.124721 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.124708 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret podName:275c0b51-3ae0-4606-8672-f047fa541200 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:48.124691343 +0000 UTC m=+10.109539798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret") pod "global-pull-secret-syncer-jqxc9" (UID: "275c0b51-3ae0-4606-8672-f047fa541200") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:44.125097 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.125078 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:44.125399 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.125384 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs podName:d719beb6-22cd-4784-a0bb-463c9a856741 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:48.125364742 +0000 UTC m=+10.110213196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs") pod "network-metrics-daemon-kzrz6" (UID: "d719beb6-22cd-4784-a0bb-463c9a856741") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:44.326178 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:44.326141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:44.326373 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.326341 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:48:44.326431 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.326374 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:48:44.326431 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.326388 2568 projected.go:196] Error preparing data for projected volume kube-api-access-h5qrp for pod openshift-network-diagnostics/network-check-target-mv8d9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:44.326536 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.326463 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp podName:5e34485f-5517-4209-ba1b-44ff7ebae506 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:48.326444469 +0000 UTC m=+10.311292931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5qrp" (UniqueName: "kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp") pod "network-check-target-mv8d9" (UID: "5e34485f-5517-4209-ba1b-44ff7ebae506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:44.598990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:44.598225 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:44.598990 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.598361 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:44.598990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:44.598710 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:44.598990 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.598795 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:44.599912 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:44.599393 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:44.599912 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:44.599515 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:46.600852 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:46.600011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:46.600852 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:46.600156 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:46.600852 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:46.600525 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:46.600852 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:46.600609 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:46.600852 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:46.600671 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:46.600852 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:46.600738 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:48.158205 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:48.158168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:48.158666 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:48.158254 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:48.158666 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.158362 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:48.158666 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.158398 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:48.158666 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.158433 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret podName:275c0b51-3ae0-4606-8672-f047fa541200 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:56.15841242 +0000 UTC m=+18.143260872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret") pod "global-pull-secret-syncer-jqxc9" (UID: "275c0b51-3ae0-4606-8672-f047fa541200") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:48.158666 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.158453 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs podName:d719beb6-22cd-4784-a0bb-463c9a856741 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:56.158442461 +0000 UTC m=+18.143290915 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs") pod "network-metrics-daemon-kzrz6" (UID: "d719beb6-22cd-4784-a0bb-463c9a856741") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:48.360592 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:48.360556 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:48.360771 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.360751 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:48:48.360850 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.360779 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:48:48.360850 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.360792 2568 projected.go:196] Error preparing data for projected volume kube-api-access-h5qrp for pod openshift-network-diagnostics/network-check-target-mv8d9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:48.360948 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.360847 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp podName:5e34485f-5517-4209-ba1b-44ff7ebae506 nodeName:}" failed. No retries permitted until 2026-07-07 12:48:56.360827525 +0000 UTC m=+18.345675978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5qrp" (UniqueName: "kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp") pod "network-check-target-mv8d9" (UID: "5e34485f-5517-4209-ba1b-44ff7ebae506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:48.598691 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:48.598619 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:48.598840 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.598732 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:48.599031 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:48.599011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:48.599144 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.599108 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:48.602324 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:48.602296 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:48.602491 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:48.602466 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:50.598014 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:50.597982 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:50.598497 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:50.597987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:50.598497 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:50.598132 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:50.598497 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:50.598146 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:50.598497 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:50.598224 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:50.598497 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:50.598308 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:52.597796 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:52.597754 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:52.598206 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:52.597900 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:52.598206 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:52.597956 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:52.598206 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:52.597987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:52.598206 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:52.598090 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:52.598447 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:52.598228 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:54.598149 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:54.598101 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:54.598585 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:54.598106 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:54.598585 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:54.598245 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:54.598585 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:54.598107 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:54.598585 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:54.598330 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:54.598585 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:54.598416 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:56.218929 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:56.218847 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:56.219402 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:56.218942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:56.219402 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.219003 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:56.219402 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.219068 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret podName:275c0b51-3ae0-4606-8672-f047fa541200 nodeName:}" failed. No retries permitted until 2026-07-07 12:49:12.219051658 +0000 UTC m=+34.203900108 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret") pod "global-pull-secret-syncer-jqxc9" (UID: "275c0b51-3ae0-4606-8672-f047fa541200") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:48:56.219402 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.219079 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:56.219402 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.219146 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs podName:d719beb6-22cd-4784-a0bb-463c9a856741 nodeName:}" failed. No retries permitted until 2026-07-07 12:49:12.219127825 +0000 UTC m=+34.203976292 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs") pod "network-metrics-daemon-kzrz6" (UID: "d719beb6-22cd-4784-a0bb-463c9a856741") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:48:56.420770 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:56.420674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:56.420927 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.420861 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:48:56.420927 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.420892 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:48:56.420927 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.420905 2568 projected.go:196] Error preparing data for projected volume kube-api-access-h5qrp for pod openshift-network-diagnostics/network-check-target-mv8d9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:56.421060 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.420964 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp podName:5e34485f-5517-4209-ba1b-44ff7ebae506 nodeName:}" failed. No retries permitted until 2026-07-07 12:49:12.420945661 +0000 UTC m=+34.405794134 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5qrp" (UniqueName: "kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp") pod "network-check-target-mv8d9" (UID: "5e34485f-5517-4209-ba1b-44ff7ebae506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:48:56.598076 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:56.597992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:56.598225 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:56.597993 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:56.598225 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.598139 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:56.598355 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.598229 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:48:56.598355 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:56.597993 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:56.598422 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:56.598363 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:58.598459 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:58.598421 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:48:58.598924 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:58.598530 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:48:58.598924 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:48:58.598574 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:48:58.598924 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:58.598582 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:48:58.598924 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:58.598650 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:48:58.598924 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:48:58.598752 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:00.598106 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.597944 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:00.598810 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.597963 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:00.598810 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:00.598190 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:00.598810 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.598020 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:00.598810 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:00.598265 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:00.598810 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:00.598395 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:00.731716 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.731688 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"fee82a74016b2167d07aedfd055158fe61c621c6d604eab8ac503ba6c7dd101c"} Jul 07 12:49:00.731831 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.731754 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"da6160d68ec9d329d81760da6bcefbf346512c7303b0135c05be2b36a3a15d91"} Jul 07 12:49:00.731831 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.731786 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"8799232ecdb75b95702ff9d36dcf8226ea0e2b10cf2506aa8fabe678162297c2"} Jul 07 12:49:00.731831 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.731821 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"7a17b9c4cf3aa8cc256673058b3d2e227c8e919c75157b161d982cdaa1d7a2f0"} Jul 07 12:49:00.731971 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.731858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"1dfca1fd500dab385c3cdd8ad89dead3550ac88280ceffea8044ef913c0fc84f"} Jul 07 12:49:00.733065 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.733042 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" event={"ID":"90229356-0deb-4f7d-bc00-f8ba96b684ff","Type":"ContainerStarted","Data":"318f0617c278a9f0ad500283c9adf5c779df8a55c67ff75eee0de5d3f9de3096"} Jul 07 12:49:00.734453 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.734426 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fqg8q" event={"ID":"3fa538d7-30b0-42e6-bb5f-2233438fd583","Type":"ContainerStarted","Data":"e1fc67673e1e67c85ea46082b1eda77c700816e0c9d7c28fd4384f37c997a759"} Jul 07 12:49:00.735818 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.735799 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrwz8" event={"ID":"d45af9fd-3eb6-4979-87e3-7e084b78aad5","Type":"ContainerStarted","Data":"90ac6fce155f1166edcad56b17944b809d6efe925ee0c934b355ef3175e5f189"} Jul 07 12:49:00.737098 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.737077 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-42l45" event={"ID":"621128a0-56bf-42dc-9566-2ddd2cf5bc58","Type":"ContainerStarted","Data":"4f28e4f14a4e24b4dfe290769811bac3373f3165189f4da2e86ed9716d1ae9c9"} Jul 07 12:49:00.738402 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.738383 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" event={"ID":"4a7301a6-3201-4be0-a95d-ac453a6ff1f5","Type":"ContainerStarted","Data":"646dd25618358753faa2d8ded58fce0b3c72be9d16f9c967668afc3f582e2c71"} Jul 07 12:49:00.739568 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.739537 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9n6g2" event={"ID":"b2be6b55-d608-438f-9bcd-2d8f9849cb16","Type":"ContainerStarted","Data":"40f1d595bb29f93bdfa71e0678b88fa96394295b79ca0c51395ad7b712b14b4a"} Jul 07 12:49:00.740924 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.740905 2568 generic.go:356] "Generic (PLEG): container finished" podID="0915ebda-b0cc-492b-91fc-1f92045f243f" containerID="56bf2626d478113df3f46934052dc7a449df3cf412c2dbf13d937b0e5765169f" exitCode=0 Jul 07 12:49:00.740993 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.740939 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" event={"ID":"0915ebda-b0cc-492b-91fc-1f92045f243f","Type":"ContainerDied","Data":"56bf2626d478113df3f46934052dc7a449df3cf412c2dbf13d937b0e5765169f"} Jul 07 12:49:00.751007 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.750976 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dxd2p" podStartSLOduration=4.385722515 podStartE2EDuration="22.750962501s" totalImagesPullingTime="18.365239986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:00.750432303 +0000 UTC m=+22.735280775" watchObservedRunningTime="2026-07-07 12:49:00.750962501 +0000 UTC m=+22.735810973" Jul 07 12:49:00.767894 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.767860 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qrwz8" podStartSLOduration=4.400906022 podStartE2EDuration="22.767848531s" totalImagesPullingTime="18.366942509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:00.767179152 +0000 UTC m=+22.752027625" watchObservedRunningTime="2026-07-07 12:49:00.767848531 +0000 UTC m=+22.752697044" Jul 07 12:49:00.782948 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.782929 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:49:00.832223 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.832205 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-42l45_621128a0-56bf-42dc-9566-2ddd2cf5bc58/node-ca/0.log" Jul 07 12:49:00.840606 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.840564 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9n6g2" podStartSLOduration=4.464325393 podStartE2EDuration="22.840547837s" totalImagesPullingTime="18.376222444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:00.818517852 +0000 UTC m=+22.803366336" watchObservedRunningTime="2026-07-07 12:49:00.840547837 +0000 UTC m=+22.825396311" Jul 07 12:49:00.841020 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.840990 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-42l45" podStartSLOduration=4.472597782 podStartE2EDuration="22.840982517s" totalImagesPullingTime="18.368384735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:00.840857696 +0000 UTC m=+22.825706160" watchObservedRunningTime="2026-07-07 12:49:00.840982517 +0000 UTC m=+22.825830990" Jul 07 12:49:00.861037 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:00.860961 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fqg8q" podStartSLOduration=4.447398581 podStartE2EDuration="22.860948908s" totalImagesPullingTime="18.413550327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:00.860277624 +0000 UTC m=+22.845126097" watchObservedRunningTime="2026-07-07 12:49:00.860948908 +0000 UTC m=+22.845797375" Jul 07 12:49:01.507259 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:01.507235 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:49:01.507863 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:01.507838 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:49:01.621588 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:01.621569 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 12:49:01.745270 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:01.745191 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" event={"ID":"4a7301a6-3201-4be0-a95d-ac453a6ff1f5","Type":"ContainerStarted","Data":"ac4f33fe1117db1321338049b0456f443f0835df89a88a78e642bef78ab5ad40"} Jul 07 12:49:01.746679 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:01.746658 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l8j8h" event={"ID":"c097b54b-f582-4f51-a99c-308303239efe","Type":"ContainerStarted","Data":"d0f540b054a5bee5209445fff699fcbf60826fe7e4d57e3a986edc130b9d6e4a"} Jul 07 12:49:01.749421 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:01.749398 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"a80318e1a04548aed5055a89ff77b5b3d31eb0819de30ed0c310805dda70a0ae"} Jul 07 12:49:01.750474 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:01.750457 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qrwz8" Jul 07 12:49:01.770887 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:01.770847 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l8j8h" podStartSLOduration=5.391569717 podStartE2EDuration="23.770834509s" totalImagesPullingTime="18.379264792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:01.770572485 +0000 UTC m=+23.755420956" watchObservedRunningTime="2026-07-07 12:49:01.770834509 +0000 UTC m=+23.755683040" Jul 07 12:49:02.516398 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:02.516288 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T12:49:01.621584337Z","UUID":"a5a06922-ffcf-4a92-851c-0e4256391fab","Handler":null,"Name":"","Endpoint":""} Jul 07 12:49:02.517776 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:02.517747 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 12:49:02.517913 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:02.517795 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 12:49:02.597315 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:02.597275 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:02.597488 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:02.597292 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:02.597488 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:02.597399 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:02.597488 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:02.597414 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:02.597488 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:02.597462 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:02.597692 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:02.597554 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:03.756968 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:03.756716 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" event={"ID":"4a7301a6-3201-4be0-a95d-ac453a6ff1f5","Type":"ContainerStarted","Data":"33f6851275c3b5d69be9b00f919fb2eff3583e15de8af200cfd0774b01275591"} Jul 07 12:49:03.759912 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:03.759888 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"2dd4a16cbd9d0c29515bcb7024e3136c1a6939e0e5eea6c9308e849431da24e6"} Jul 07 12:49:03.805276 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:03.805235 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp5lk" podStartSLOduration=5.213378872 podStartE2EDuration="25.805221362s" totalImagesPullingTime="20.59184249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:03.801160201 +0000 UTC m=+25.786008674" watchObservedRunningTime="2026-07-07 12:49:03.805221362 +0000 UTC m=+25.790069834" Jul 07 12:49:04.597696 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:04.597671 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:04.598724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:04.597698 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:04.598724 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:04.597799 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:04.598724 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:04.597866 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:04.598724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:04.597895 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:04.598724 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:04.597996 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:06.598071 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.597865 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:06.598738 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.597872 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:06.598738 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:06.598101 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:06.598738 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.597884 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:06.598738 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:06.598208 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:06.598738 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:06.598280 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:06.767651 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.767624 2568 generic.go:356] "Generic (PLEG): container finished" podID="0915ebda-b0cc-492b-91fc-1f92045f243f" containerID="1862e3b4bd9f3d8394c4b2a847b8dd5ea36e3296adbc4c88714eae53af3930d7" exitCode=0 Jul 07 12:49:06.767803 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.767705 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" event={"ID":"0915ebda-b0cc-492b-91fc-1f92045f243f","Type":"ContainerDied","Data":"1862e3b4bd9f3d8394c4b2a847b8dd5ea36e3296adbc4c88714eae53af3930d7"} Jul 07 12:49:06.771047 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.771020 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" event={"ID":"1542a9c1-a15f-4413-acc5-fff6edbadd71","Type":"ContainerStarted","Data":"88ed8c50da7ed4e81063ec5ec621f1c4177ab495a2947b49c56bf04f1da08df1"} Jul 07 12:49:06.771277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.771258 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:49:06.771342 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.771335 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:49:06.786641 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.786624 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:49:06.844558 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:06.844520 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" podStartSLOduration=10.396502899 podStartE2EDuration="28.844505138s" totalImagesPullingTime="18.448002239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:06.843819092 +0000 UTC m=+28.828667564" watchObservedRunningTime="2026-07-07 12:49:06.844505138 +0000 UTC m=+28.829353609" Jul 07 12:49:07.772718 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:07.772694 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:49:07.789840 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:07.789821 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:49:08.597826 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:08.597795 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:08.598007 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:08.597911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:08.598007 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:08.597939 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:08.598129 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:08.597905 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:08.598129 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:08.598035 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:08.598204 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:08.598126 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:08.776397 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:08.776372 2568 generic.go:356] "Generic (PLEG): container finished" podID="0915ebda-b0cc-492b-91fc-1f92045f243f" containerID="9faf2b3dd670b9d7c0820e2015f634fd6c2da632f9bcf8d6170244510abd9b20" exitCode=0 Jul 07 12:49:08.776813 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:08.776443 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" event={"ID":"0915ebda-b0cc-492b-91fc-1f92045f243f","Type":"ContainerDied","Data":"9faf2b3dd670b9d7c0820e2015f634fd6c2da632f9bcf8d6170244510abd9b20"} Jul 07 12:49:10.597765 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:10.597738 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:10.598166 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:10.597738 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:10.598166 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:10.597870 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:10.598166 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:10.597748 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:10.598166 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:10.597904 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:10.598166 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:10.598004 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:10.781726 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:10.781700 2568 generic.go:356] "Generic (PLEG): container finished" podID="0915ebda-b0cc-492b-91fc-1f92045f243f" containerID="322d2994fa99c6237cb6574f1ee65b56bdadac20ecdc6138ec127bcc8b73938e" exitCode=0 Jul 07 12:49:10.781896 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:10.781751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" event={"ID":"0915ebda-b0cc-492b-91fc-1f92045f243f","Type":"ContainerDied","Data":"322d2994fa99c6237cb6574f1ee65b56bdadac20ecdc6138ec127bcc8b73938e"} Jul 07 12:49:12.251056 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:12.251024 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:12.251507 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:12.251096 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:12.251507 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.251206 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:49:12.251507 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.251265 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret podName:275c0b51-3ae0-4606-8672-f047fa541200 nodeName:}" failed. No retries permitted until 2026-07-07 12:49:44.251250918 +0000 UTC m=+66.236099368 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret") pod "global-pull-secret-syncer-jqxc9" (UID: "275c0b51-3ae0-4606-8672-f047fa541200") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:49:12.251507 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.251211 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:49:12.251507 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.251350 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs podName:d719beb6-22cd-4784-a0bb-463c9a856741 nodeName:}" failed. No retries permitted until 2026-07-07 12:49:44.251331752 +0000 UTC m=+66.236180219 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs") pod "network-metrics-daemon-kzrz6" (UID: "d719beb6-22cd-4784-a0bb-463c9a856741") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:49:12.453199 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:12.453164 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:12.453369 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.453342 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:49:12.453369 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.453368 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:49:12.453513 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.453380 2568 projected.go:196] Error preparing data for projected volume kube-api-access-h5qrp for pod openshift-network-diagnostics/network-check-target-mv8d9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:49:12.453513 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.453436 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp podName:5e34485f-5517-4209-ba1b-44ff7ebae506 nodeName:}" failed. No retries permitted until 2026-07-07 12:49:44.453418182 +0000 UTC m=+66.438266637 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5qrp" (UniqueName: "kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp") pod "network-check-target-mv8d9" (UID: "5e34485f-5517-4209-ba1b-44ff7ebae506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:49:12.597529 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:12.597457 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:12.597663 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:12.597464 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:12.597663 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.597558 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:12.597663 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:12.597464 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:12.597663 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.597626 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:12.597786 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:12.597693 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:14.597728 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:14.597698 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:14.598154 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:14.597703 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:14.598154 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:14.597837 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:14.598154 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:14.597705 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:14.598154 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:14.597894 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:14.598154 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:14.597944 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:15.795744 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:15.795668 2568 generic.go:356] "Generic (PLEG): container finished" podID="0915ebda-b0cc-492b-91fc-1f92045f243f" containerID="f905f00bda407c7ffc1e292264897bf9a3962a39a80cde4ea1ee3b41db0fdeec" exitCode=0 Jul 07 12:49:15.795744 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:15.795728 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" event={"ID":"0915ebda-b0cc-492b-91fc-1f92045f243f","Type":"ContainerDied","Data":"f905f00bda407c7ffc1e292264897bf9a3962a39a80cde4ea1ee3b41db0fdeec"} Jul 07 12:49:16.598024 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:16.597995 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:16.598209 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:16.597995 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:16.598209 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:16.598091 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:16.598209 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:16.598125 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:16.598404 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:16.598209 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:16.598404 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:16.598259 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:16.801197 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:16.801164 2568 generic.go:356] "Generic (PLEG): container finished" podID="0915ebda-b0cc-492b-91fc-1f92045f243f" containerID="2d7a93655186ceba8e6a46bbc21ca07ef82b056267635a5e5c5b2229acb4bf65" exitCode=0 Jul 07 12:49:16.801522 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:16.801224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" event={"ID":"0915ebda-b0cc-492b-91fc-1f92045f243f","Type":"ContainerDied","Data":"2d7a93655186ceba8e6a46bbc21ca07ef82b056267635a5e5c5b2229acb4bf65"} Jul 07 12:49:17.806971 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:17.806778 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" event={"ID":"0915ebda-b0cc-492b-91fc-1f92045f243f","Type":"ContainerStarted","Data":"253011fcdb28eb200fea91e447a39f0fd309fbd3d49ce4d13529810cfa872194"} Jul 07 12:49:18.598181 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:18.598147 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:18.598181 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:18.598182 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:18.598426 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:18.598250 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:18.598426 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:18.598300 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:18.598426 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:18.598333 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:18.598426 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:18.598392 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:20.597960 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:20.597930 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:20.598378 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:20.597920 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:20.598378 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:20.598051 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:20.598378 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:20.598163 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:20.598378 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:20.597939 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:20.598378 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:20.598249 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:22.597388 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:22.597359 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:22.597762 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:22.597396 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:22.597762 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:22.597359 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:22.597762 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:22.597449 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:22.597762 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:22.597522 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:22.597762 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:22.597601 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:24.597876 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:24.597851 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:24.597876 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:24.597882 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:24.598472 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:24.597972 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:24.598472 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:24.598001 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:24.598472 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:24.598044 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:24.598472 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:24.598135 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:26.597284 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:26.597254 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:26.597765 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:26.597353 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:26.597765 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:26.597256 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:26.597765 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:26.597402 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:26.597765 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:26.597518 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:26.597765 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:26.597583 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:28.598592 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.598548 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:28.599241 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:28.598631 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:28.599241 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.598733 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:28.599241 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:28.598844 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:28.599241 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.598896 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:28.599241 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:28.598966 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:28.778665 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.778555 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tcbbf" podStartSLOduration=20.598372215 podStartE2EDuration="50.778535022s" totalImagesPullingTime="30.180162807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:17.852393471 +0000 UTC m=+39.837241943" watchObservedRunningTime="2026-07-07 12:49:28.778535022 +0000 UTC m=+50.763383495" Jul 07 12:49:28.779270 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.779247 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mv8d9"] Jul 07 12:49:28.780768 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.780745 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jqxc9"] Jul 07 12:49:28.781259 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.781241 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kzrz6"] Jul 07 12:49:28.828005 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.827987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:28.828129 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.827987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:28.828129 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:28.828095 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:28.828229 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:28.827987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:28.828229 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:28.828177 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:28.828229 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:28.828213 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:30.597907 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:30.597877 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:30.597907 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:30.597896 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:30.598372 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:30.597887 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:30.598372 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:30.597974 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:30.598372 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:30.598041 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:30.598372 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:30.598162 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:32.597655 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.597475 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:32.598219 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.597546 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:32.598219 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:32.597737 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mv8d9" podUID="5e34485f-5517-4209-ba1b-44ff7ebae506" Jul 07 12:49:32.598219 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.597571 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:32.598219 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:32.597839 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jqxc9" podUID="275c0b51-3ae0-4606-8672-f047fa541200" Jul 07 12:49:32.598219 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:32.597911 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kzrz6" podUID="d719beb6-22cd-4784-a0bb-463c9a856741" Jul 07 12:49:32.681978 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.681959 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-17.ec2.internal" event="NodeReady" Jul 07 12:49:32.682082 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.682065 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 12:49:32.762457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.762437 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w7t4f"] Jul 07 12:49:32.781997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.781970 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-shzbb"] Jul 07 12:49:32.782128 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.782093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:32.784885 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.784856 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 12:49:32.784991 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.784945 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wb7sm\"" Jul 07 12:49:32.785218 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.785200 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 12:49:32.795554 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.795535 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w7t4f"] Jul 07 12:49:32.795641 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.795621 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:32.796231 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.796213 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5hx9w"] Jul 07 12:49:32.799416 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.799398 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 12:49:32.799500 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.799484 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 12:49:32.799541 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.799510 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 12:49:32.799741 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.799725 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 12:49:32.799793 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.799733 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2zm52\"" Jul 07 12:49:32.808473 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.808457 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5hx9w" Jul 07 12:49:32.811846 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.811830 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 12:49:32.812127 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.812096 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-58r4b\"" Jul 07 12:49:32.812345 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.812320 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-shzbb"] Jul 07 12:49:32.812446 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.812386 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 12:49:32.812913 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.812892 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 12:49:32.813732 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.813710 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5hx9w"] Jul 07 12:49:32.907552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/badecb7c-59ef-4bc4-a8eb-4817217e8409-data-volume\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:32.907552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907519 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/badecb7c-59ef-4bc4-a8eb-4817217e8409-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:32.907552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907543 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9cb8c914-5ba9-4e15-a054-01d42725ad6d-config-volume\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:32.907788 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907566 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cb8c914-5ba9-4e15-a054-01d42725ad6d-tmp-dir\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:32.907788 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907652 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9-cert\") pod \"ingress-canary-5hx9w\" (UID: \"dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9\") " pod="openshift-ingress-canary/ingress-canary-5hx9w" Jul 07 12:49:32.907788 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/badecb7c-59ef-4bc4-a8eb-4817217e8409-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:32.907788 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907716 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55bl8\" (UniqueName: \"kubernetes.io/projected/badecb7c-59ef-4bc4-a8eb-4817217e8409-kube-api-access-55bl8\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:32.907788 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907747 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s9xb\" (UniqueName: \"kubernetes.io/projected/dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9-kube-api-access-4s9xb\") pod \"ingress-canary-5hx9w\" (UID: \"dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9\") " pod="openshift-ingress-canary/ingress-canary-5hx9w" Jul 07 12:49:32.907788 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907767 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9cb8c914-5ba9-4e15-a054-01d42725ad6d-metrics-tls\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:32.907975 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907793 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9bbq\" (UniqueName: \"kubernetes.io/projected/9cb8c914-5ba9-4e15-a054-01d42725ad6d-kube-api-access-d9bbq\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:32.907975 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:32.907824 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/badecb7c-59ef-4bc4-a8eb-4817217e8409-crio-socket\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.008861 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.008830 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/badecb7c-59ef-4bc4-a8eb-4817217e8409-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.009031 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.008865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55bl8\" (UniqueName: \"kubernetes.io/projected/badecb7c-59ef-4bc4-a8eb-4817217e8409-kube-api-access-55bl8\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.009031 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.008886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s9xb\" (UniqueName: \"kubernetes.io/projected/dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9-kube-api-access-4s9xb\") pod \"ingress-canary-5hx9w\" (UID: \"dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9\") " pod="openshift-ingress-canary/ingress-canary-5hx9w" Jul 07 12:49:33.009031 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.008910 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9cb8c914-5ba9-4e15-a054-01d42725ad6d-metrics-tls\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.009031 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.008947 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9bbq\" (UniqueName: \"kubernetes.io/projected/9cb8c914-5ba9-4e15-a054-01d42725ad6d-kube-api-access-d9bbq\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.009031 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.008997 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/badecb7c-59ef-4bc4-a8eb-4817217e8409-crio-socket\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.009329 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/badecb7c-59ef-4bc4-a8eb-4817217e8409-data-volume\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.009329 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/badecb7c-59ef-4bc4-a8eb-4817217e8409-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.009329 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009188 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9cb8c914-5ba9-4e15-a054-01d42725ad6d-config-volume\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.009329 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009211 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cb8c914-5ba9-4e15-a054-01d42725ad6d-tmp-dir\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.009329 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009218 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/badecb7c-59ef-4bc4-a8eb-4817217e8409-crio-socket\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.009329 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009242 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9-cert\") pod \"ingress-canary-5hx9w\" (UID: \"dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9\") " pod="openshift-ingress-canary/ingress-canary-5hx9w" Jul 07 12:49:33.009630 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009419 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/badecb7c-59ef-4bc4-a8eb-4817217e8409-data-volume\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.009630 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009538 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/badecb7c-59ef-4bc4-a8eb-4817217e8409-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.009630 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009577 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cb8c914-5ba9-4e15-a054-01d42725ad6d-tmp-dir\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.009862 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.009835 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9cb8c914-5ba9-4e15-a054-01d42725ad6d-config-volume\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.012702 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.012684 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9cb8c914-5ba9-4e15-a054-01d42725ad6d-metrics-tls\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.012780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.012688 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/badecb7c-59ef-4bc4-a8eb-4817217e8409-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.012780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.012765 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9-cert\") pod \"ingress-canary-5hx9w\" (UID: \"dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9\") " pod="openshift-ingress-canary/ingress-canary-5hx9w" Jul 07 12:49:33.025420 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.025402 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s9xb\" (UniqueName: \"kubernetes.io/projected/dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9-kube-api-access-4s9xb\") pod \"ingress-canary-5hx9w\" (UID: \"dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9\") " pod="openshift-ingress-canary/ingress-canary-5hx9w" Jul 07 12:49:33.027649 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.027620 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9bbq\" (UniqueName: \"kubernetes.io/projected/9cb8c914-5ba9-4e15-a054-01d42725ad6d-kube-api-access-d9bbq\") pod \"dns-default-w7t4f\" (UID: \"9cb8c914-5ba9-4e15-a054-01d42725ad6d\") " pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.027720 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.027671 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55bl8\" (UniqueName: \"kubernetes.io/projected/badecb7c-59ef-4bc4-a8eb-4817217e8409-kube-api-access-55bl8\") pod \"insights-runtime-extractor-shzbb\" (UID: \"badecb7c-59ef-4bc4-a8eb-4817217e8409\") " pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.092774 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.092745 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:33.103780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.103759 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-shzbb" Jul 07 12:49:33.117718 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.117693 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5hx9w" Jul 07 12:49:33.264628 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.264574 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-shzbb"] Jul 07 12:49:33.268348 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:33.268321 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbadecb7c_59ef_4bc4_a8eb_4817217e8409.slice/crio-4503d28c44020086e2572cd0196e1f734ca0f5af177003b5d158b85556bff31e WatchSource:0}: Error finding container 4503d28c44020086e2572cd0196e1f734ca0f5af177003b5d158b85556bff31e: Status 404 returned error can't find the container with id 4503d28c44020086e2572cd0196e1f734ca0f5af177003b5d158b85556bff31e Jul 07 12:49:33.272639 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.272616 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w7t4f"] Jul 07 12:49:33.277801 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:33.277779 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb8c914_5ba9_4e15_a054_01d42725ad6d.slice/crio-53234410a9064b2f95da7feddf506e3b2c0a2c98229f9ab2e16639c120cfc0be WatchSource:0}: Error finding container 53234410a9064b2f95da7feddf506e3b2c0a2c98229f9ab2e16639c120cfc0be: Status 404 returned error can't find the container with id 53234410a9064b2f95da7feddf506e3b2c0a2c98229f9ab2e16639c120cfc0be Jul 07 12:49:33.294090 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.294069 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5hx9w"] Jul 07 12:49:33.296705 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:33.296684 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd5eed6_adfa_4d1a_ae8c_3f2189b774f9.slice/crio-4478df221aa94d8fae2dace64f00baf66d25dbcdee3b9f5f413afef2d600159a WatchSource:0}: Error finding container 4478df221aa94d8fae2dace64f00baf66d25dbcdee3b9f5f413afef2d600159a: Status 404 returned error can't find the container with id 4478df221aa94d8fae2dace64f00baf66d25dbcdee3b9f5f413afef2d600159a Jul 07 12:49:33.842921 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.842833 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5hx9w" event={"ID":"dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9","Type":"ContainerStarted","Data":"4478df221aa94d8fae2dace64f00baf66d25dbcdee3b9f5f413afef2d600159a"} Jul 07 12:49:33.845949 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.845840 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shzbb" event={"ID":"badecb7c-59ef-4bc4-a8eb-4817217e8409","Type":"ContainerStarted","Data":"4503d28c44020086e2572cd0196e1f734ca0f5af177003b5d158b85556bff31e"} Jul 07 12:49:33.847409 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:33.847356 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w7t4f" event={"ID":"9cb8c914-5ba9-4e15-a054-01d42725ad6d","Type":"ContainerStarted","Data":"53234410a9064b2f95da7feddf506e3b2c0a2c98229f9ab2e16639c120cfc0be"} Jul 07 12:49:34.503548 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.503511 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86d4f56489-h4jcz"] Jul 07 12:49:34.505919 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.505888 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.510605 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.510583 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 12:49:34.510718 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.510588 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 12:49:34.511837 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.511817 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 12:49:34.512061 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.512043 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 12:49:34.513938 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.512828 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 12:49:34.513938 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.513692 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 12:49:34.513938 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.513730 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4bbkw\"" Jul 07 12:49:34.514531 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.514512 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 12:49:34.522368 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.522343 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 12:49:34.527614 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.527595 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d4f56489-h4jcz"] Jul 07 12:49:34.602588 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.602556 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:34.602917 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.602903 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:34.603284 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.603267 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:34.606548 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.606524 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cw2d4\"" Jul 07 12:49:34.606824 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.606806 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 12:49:34.607172 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.607154 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 12:49:34.607382 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.607366 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g5zg7\"" Jul 07 12:49:34.607592 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.607524 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 12:49:34.607764 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.607738 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 12:49:34.624887 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.624864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-service-ca\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.624986 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.624909 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-oauth-config\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.624986 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.624937 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-trusted-ca-bundle\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.624986 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.624962 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-oauth-serving-cert\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.625151 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.625022 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thjhr\" (UniqueName: \"kubernetes.io/projected/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-kube-api-access-thjhr\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.625151 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.625059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-config\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.625151 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.625131 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-serving-cert\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.725571 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.725521 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-service-ca\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.725724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.725592 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-oauth-config\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.725724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.725621 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-trusted-ca-bundle\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.725724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.725645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-oauth-serving-cert\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.725724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.725674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thjhr\" (UniqueName: \"kubernetes.io/projected/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-kube-api-access-thjhr\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.725724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.725702 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-config\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.725969 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.725739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-serving-cert\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.726346 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.726315 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-service-ca\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.726737 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.726714 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-oauth-serving-cert\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.728194 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.727558 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-trusted-ca-bundle\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.728194 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.728018 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-config\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.730088 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.730064 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-serving-cert\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.730214 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.730135 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-oauth-config\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.736322 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.736281 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thjhr\" (UniqueName: \"kubernetes.io/projected/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-kube-api-access-thjhr\") pod \"console-86d4f56489-h4jcz\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:34.819018 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:34.818948 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:36.887610 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.887573 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b"] Jul 07 12:49:36.890593 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.890567 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:36.899261 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.899228 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xcjbm\"" Jul 07 12:49:36.899261 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.899240 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 12:49:36.899417 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.899243 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 12:49:36.899417 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.899330 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 12:49:36.899417 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.899246 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 12:49:36.899417 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.899374 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 12:49:36.912960 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:36.912932 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b"] Jul 07 12:49:37.045249 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.045218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.045249 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.045259 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0358bae-c731-4629-b7c4-6929f09fc980-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.045484 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.045281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfq85\" (UniqueName: \"kubernetes.io/projected/e0358bae-c731-4629-b7c4-6929f09fc980-kube-api-access-rfq85\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.045484 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.045305 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.146138 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.146037 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.146138 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.146099 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0358bae-c731-4629-b7c4-6929f09fc980-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.146380 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.146144 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfq85\" (UniqueName: \"kubernetes.io/projected/e0358bae-c731-4629-b7c4-6929f09fc980-kube-api-access-rfq85\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.146380 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.146178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.146380 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:37.146219 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 07 12:49:37.146380 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:37.146301 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-tls podName:e0358bae-c731-4629-b7c4-6929f09fc980 nodeName:}" failed. No retries permitted until 2026-07-07 12:49:37.646272337 +0000 UTC m=+59.631120855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-zhz9b" (UID: "e0358bae-c731-4629-b7c4-6929f09fc980") : secret "prometheus-operator-tls" not found Jul 07 12:49:37.146942 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.146913 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0358bae-c731-4629-b7c4-6929f09fc980-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.148891 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.148871 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.157664 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.157634 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfq85\" (UniqueName: \"kubernetes.io/projected/e0358bae-c731-4629-b7c4-6929f09fc980-kube-api-access-rfq85\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.650800 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.650768 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.653363 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.653332 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0358bae-c731-4629-b7c4-6929f09fc980-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-zhz9b\" (UID: \"e0358bae-c731-4629-b7c4-6929f09fc980\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.801017 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.800987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" Jul 07 12:49:37.993381 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:37.993345 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d4f56489-h4jcz"] Jul 07 12:49:38.207558 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:38.207532 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8de0b7_7e1d_45b7_80ac_e46734bb1ab5.slice/crio-92e230a7049903e72a4f184d678702f5dff6c8b1c7e4292f37886a6a006fcd88 WatchSource:0}: Error finding container 92e230a7049903e72a4f184d678702f5dff6c8b1c7e4292f37886a6a006fcd88: Status 404 returned error can't find the container with id 92e230a7049903e72a4f184d678702f5dff6c8b1c7e4292f37886a6a006fcd88 Jul 07 12:49:38.354721 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.354585 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b"] Jul 07 12:49:38.360486 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:38.360451 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0358bae_c731_4629_b7c4_6929f09fc980.slice/crio-eefe3ea34242615d0b6a189ae1e3952902c9e1538ba9af3d11fbd8251e96b1cc WatchSource:0}: Error finding container eefe3ea34242615d0b6a189ae1e3952902c9e1538ba9af3d11fbd8251e96b1cc: Status 404 returned error can't find the container with id eefe3ea34242615d0b6a189ae1e3952902c9e1538ba9af3d11fbd8251e96b1cc Jul 07 12:49:38.862714 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.862677 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w7t4f" event={"ID":"9cb8c914-5ba9-4e15-a054-01d42725ad6d","Type":"ContainerStarted","Data":"ebc499a26f898327aac32d1e5488d74e547cca9fa901f1481e81ea82523f8778"} Jul 07 12:49:38.862899 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.862742 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w7t4f" event={"ID":"9cb8c914-5ba9-4e15-a054-01d42725ad6d","Type":"ContainerStarted","Data":"f759175b061735d75136850512f2ed5778da46ae776a87f3ddc2810395d37fe2"} Jul 07 12:49:38.862899 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.862793 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:38.864554 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.864527 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d4f56489-h4jcz" event={"ID":"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5","Type":"ContainerStarted","Data":"92e230a7049903e72a4f184d678702f5dff6c8b1c7e4292f37886a6a006fcd88"} Jul 07 12:49:38.866292 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.866237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5hx9w" event={"ID":"dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9","Type":"ContainerStarted","Data":"5d31bfabc65d37e2115f9d9b0a40f52c9389d9cea5cfcdf3c47fcaa526b3451d"} Jul 07 12:49:38.868147 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.868106 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shzbb" event={"ID":"badecb7c-59ef-4bc4-a8eb-4817217e8409","Type":"ContainerStarted","Data":"ad6c2ddf49f174ef3f89bd281d6b1b534d07e68bcbdb4f1769dc09be7d6abeb4"} Jul 07 12:49:38.869449 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.869399 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" event={"ID":"e0358bae-c731-4629-b7c4-6929f09fc980","Type":"ContainerStarted","Data":"eefe3ea34242615d0b6a189ae1e3952902c9e1538ba9af3d11fbd8251e96b1cc"} Jul 07 12:49:38.918799 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.918764 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5hx9w" podStartSLOduration=2.011457354 podStartE2EDuration="6.918752404s" totalImagesPullingTime="4.90729505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:38.918496743 +0000 UTC m=+60.903345226" watchObservedRunningTime="2026-07-07 12:49:38.918752404 +0000 UTC m=+60.903600876" Jul 07 12:49:38.919486 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:38.919457 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w7t4f" podStartSLOduration=1.993372338 podStartE2EDuration="6.919443515s" totalImagesPullingTime="4.926071177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:38.891291019 +0000 UTC m=+60.876139491" watchObservedRunningTime="2026-07-07 12:49:38.919443515 +0000 UTC m=+60.904291966" Jul 07 12:49:39.797325 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:39.797288 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hjrd" Jul 07 12:49:39.874616 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:39.874567 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shzbb" event={"ID":"badecb7c-59ef-4bc4-a8eb-4817217e8409","Type":"ContainerStarted","Data":"145d56b26dfd81715cb3c5696c269260a9ab2008aacd288e23b31e9f8dd89d50"} Jul 07 12:49:42.884629 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:42.884589 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d4f56489-h4jcz" event={"ID":"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5","Type":"ContainerStarted","Data":"793f8adf71135e70159b76700667128041f0ab6311b297a719d9b0bc457f7519"} Jul 07 12:49:42.890083 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:42.890052 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shzbb" event={"ID":"badecb7c-59ef-4bc4-a8eb-4817217e8409","Type":"ContainerStarted","Data":"e62d42b95107149ea7fa11f430153ffc8c37dd0d413b16e80b98a9813bb9656e"} Jul 07 12:49:42.891773 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:42.891751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" event={"ID":"e0358bae-c731-4629-b7c4-6929f09fc980","Type":"ContainerStarted","Data":"b191bf666c08756373098d33f97ca67039dceef7ab8b26d753e68275c7923d1e"} Jul 07 12:49:42.891879 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:42.891786 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" event={"ID":"e0358bae-c731-4629-b7c4-6929f09fc980","Type":"ContainerStarted","Data":"798a720b44777e6c27a9f7c8014b48d4caedd84cd9c13fb0dfcefe0679d059ec"} Jul 07 12:49:42.969632 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:42.969577 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86d4f56489-h4jcz" podStartSLOduration=4.723912013 podStartE2EDuration="8.969562074s" totalImagesPullingTime="4.245650061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:42.924136847 +0000 UTC m=+64.908985317" watchObservedRunningTime="2026-07-07 12:49:42.969562074 +0000 UTC m=+64.954410545" Jul 07 12:49:43.007227 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:43.007173 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zhz9b" podStartSLOduration=2.922064385 podStartE2EDuration="7.007158343s" totalImagesPullingTime="4.085093958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:43.006966101 +0000 UTC m=+64.991814573" watchObservedRunningTime="2026-07-07 12:49:43.007158343 +0000 UTC m=+64.992006815" Jul 07 12:49:43.007717 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:43.007688 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-shzbb" podStartSLOduration=2.005046534 podStartE2EDuration="11.007676948s" totalImagesPullingTime="9.002630414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:49:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:42.973433153 +0000 UTC m=+64.958281637" watchObservedRunningTime="2026-07-07 12:49:43.007676948 +0000 UTC m=+64.992525419" Jul 07 12:49:44.306388 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.306350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:44.306815 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.306407 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:44.333289 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.333267 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 12:49:44.333725 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.333711 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 12:49:44.338939 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.338917 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/275c0b51-3ae0-4606-8672-f047fa541200-original-pull-secret\") pod \"global-pull-secret-syncer-jqxc9\" (UID: \"275c0b51-3ae0-4606-8672-f047fa541200\") " pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:44.339055 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.339036 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d719beb6-22cd-4784-a0bb-463c9a856741-metrics-certs\") pod \"network-metrics-daemon-kzrz6\" (UID: \"d719beb6-22cd-4784-a0bb-463c9a856741\") " pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:44.508050 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.508017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:44.511091 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.511073 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 12:49:44.518619 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.518603 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jqxc9" Jul 07 12:49:44.521696 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.521675 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 12:49:44.530192 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.530170 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cw2d4\"" Jul 07 12:49:44.531601 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.531580 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5qrp\" (UniqueName: \"kubernetes.io/projected/5e34485f-5517-4209-ba1b-44ff7ebae506-kube-api-access-h5qrp\") pod \"network-check-target-mv8d9\" (UID: \"5e34485f-5517-4209-ba1b-44ff7ebae506\") " pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:44.537710 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.537691 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kzrz6" Jul 07 12:49:44.538433 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.538413 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g5zg7\"" Jul 07 12:49:44.544827 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.544807 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:44.684672 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.684649 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jqxc9"] Jul 07 12:49:44.686994 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:44.686970 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275c0b51_3ae0_4606_8672_f047fa541200.slice/crio-1b0136843d5d807757e083981fce96e4e5d8039f1c128f2540814c7be5695f88 WatchSource:0}: Error finding container 1b0136843d5d807757e083981fce96e4e5d8039f1c128f2540814c7be5695f88: Status 404 returned error can't find the container with id 1b0136843d5d807757e083981fce96e4e5d8039f1c128f2540814c7be5695f88 Jul 07 12:49:44.719163 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.719143 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kzrz6"] Jul 07 12:49:44.721993 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:44.721970 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd719beb6_22cd_4784_a0bb_463c9a856741.slice/crio-73c3f602e22c76fa02c93093acc8e44897809cb15b398b47786290f0a36081da WatchSource:0}: Error finding container 73c3f602e22c76fa02c93093acc8e44897809cb15b398b47786290f0a36081da: Status 404 returned error can't find the container with id 73c3f602e22c76fa02c93093acc8e44897809cb15b398b47786290f0a36081da Jul 07 12:49:44.725639 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.725620 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mv8d9"] Jul 07 12:49:44.729109 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:44.729083 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e34485f_5517_4209_ba1b_44ff7ebae506.slice/crio-50e54e22f1dcbf325e28da2c28cc5d576fa5b6e9636a3aa78d76a1e86cd972a4 WatchSource:0}: Error finding container 50e54e22f1dcbf325e28da2c28cc5d576fa5b6e9636a3aa78d76a1e86cd972a4: Status 404 returned error can't find the container with id 50e54e22f1dcbf325e28da2c28cc5d576fa5b6e9636a3aa78d76a1e86cd972a4 Jul 07 12:49:44.819528 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.819470 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:44.819528 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.819516 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:49:44.820764 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.820741 2568 patch_prober.go:28] interesting pod/console-86d4f56489-h4jcz container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 07 12:49:44.820817 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.820781 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-86d4f56489-h4jcz" podUID="1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 07 12:49:44.898763 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.898734 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kzrz6" event={"ID":"d719beb6-22cd-4784-a0bb-463c9a856741","Type":"ContainerStarted","Data":"73c3f602e22c76fa02c93093acc8e44897809cb15b398b47786290f0a36081da"} Jul 07 12:49:44.899820 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.899798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jqxc9" event={"ID":"275c0b51-3ae0-4606-8672-f047fa541200","Type":"ContainerStarted","Data":"1b0136843d5d807757e083981fce96e4e5d8039f1c128f2540814c7be5695f88"} Jul 07 12:49:44.900895 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:44.900874 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mv8d9" event={"ID":"5e34485f-5517-4209-ba1b-44ff7ebae506","Type":"ContainerStarted","Data":"50e54e22f1dcbf325e28da2c28cc5d576fa5b6e9636a3aa78d76a1e86cd972a4"} Jul 07 12:49:45.594968 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.594203 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf"] Jul 07 12:49:45.597181 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.597155 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.612908 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.612865 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 12:49:45.613409 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.612870 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 12:49:45.613780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.613763 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-28zh4\"" Jul 07 12:49:45.632665 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.632637 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf"] Jul 07 12:49:45.651169 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.649958 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vcz6t"] Jul 07 12:49:45.654703 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.654685 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.658506 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.658486 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 12:49:45.658843 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.658786 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 12:49:45.659127 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.659095 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j4q47\"" Jul 07 12:49:45.659878 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.659711 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 12:49:45.716499 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.716464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7790c18-eb2c-4f59-8390-1afbbc8a793d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.716632 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.716517 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lc87\" (UniqueName: \"kubernetes.io/projected/a7790c18-eb2c-4f59-8390-1afbbc8a793d-kube-api-access-7lc87\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.716632 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.716562 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7790c18-eb2c-4f59-8390-1afbbc8a793d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.716632 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.716591 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7790c18-eb2c-4f59-8390-1afbbc8a793d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.817474 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817439 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-root\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.817621 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-textfile\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.817621 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817503 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-wtmp\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.817744 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lc87\" (UniqueName: \"kubernetes.io/projected/a7790c18-eb2c-4f59-8390-1afbbc8a793d-kube-api-access-7lc87\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.817744 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817658 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-sys\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.817744 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-accelerators-collector-config\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.817744 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7790c18-eb2c-4f59-8390-1afbbc8a793d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.817936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7790c18-eb2c-4f59-8390-1afbbc8a793d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.817936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817808 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7790c18-eb2c-4f59-8390-1afbbc8a793d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.817936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.817936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817879 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhpql\" (UniqueName: \"kubernetes.io/projected/9eb4ac60-a8e6-437d-b18b-631e3c984e70-kube-api-access-hhpql\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.817936 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817916 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9eb4ac60-a8e6-437d-b18b-631e3c984e70-metrics-client-ca\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.818187 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.817945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-tls\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.818335 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.818308 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7790c18-eb2c-4f59-8390-1afbbc8a793d-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.820627 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.820567 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7790c18-eb2c-4f59-8390-1afbbc8a793d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.821202 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.821183 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7790c18-eb2c-4f59-8390-1afbbc8a793d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.829338 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.829297 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lc87\" (UniqueName: \"kubernetes.io/projected/a7790c18-eb2c-4f59-8390-1afbbc8a793d-kube-api-access-7lc87\") pod \"openshift-state-metrics-54cf96dfcf-sbxpf\" (UID: \"a7790c18-eb2c-4f59-8390-1afbbc8a793d\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.918699 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.918808 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918707 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhpql\" (UniqueName: \"kubernetes.io/projected/9eb4ac60-a8e6-437d-b18b-631e3c984e70-kube-api-access-hhpql\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.918808 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918738 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9eb4ac60-a8e6-437d-b18b-631e3c984e70-metrics-client-ca\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.918808 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918767 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-tls\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.918808 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918795 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-root\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-textfile\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918824 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-wtmp\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-sys\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.918881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-accelerators-collector-config\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919191 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.919108 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-root\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919297 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.919213 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-wtmp\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919352 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.919295 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9eb4ac60-a8e6-437d-b18b-631e3c984e70-sys\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919480 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.919455 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-accelerators-collector-config\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919541 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.919523 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9eb4ac60-a8e6-437d-b18b-631e3c984e70-metrics-client-ca\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.919685 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.919664 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-textfile\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.921300 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.921280 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.921770 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.921748 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9eb4ac60-a8e6-437d-b18b-631e3c984e70-node-exporter-tls\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.921858 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.921845 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" Jul 07 12:49:45.958012 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.957985 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhpql\" (UniqueName: \"kubernetes.io/projected/9eb4ac60-a8e6-437d-b18b-631e3c984e70-kube-api-access-hhpql\") pod \"node-exporter-vcz6t\" (UID: \"9eb4ac60-a8e6-437d-b18b-631e3c984e70\") " pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:45.966806 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:45.966789 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vcz6t" Jul 07 12:49:46.008206 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:46.007749 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb4ac60_a8e6_437d_b18b_631e3c984e70.slice/crio-61d7fbfe242359570654678aa7b69a8b62b751fc6c9666957fb08d4795d11b7e WatchSource:0}: Error finding container 61d7fbfe242359570654678aa7b69a8b62b751fc6c9666957fb08d4795d11b7e: Status 404 returned error can't find the container with id 61d7fbfe242359570654678aa7b69a8b62b751fc6c9666957fb08d4795d11b7e Jul 07 12:49:46.143086 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.143014 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf"] Jul 07 12:49:46.154700 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:46.154675 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7790c18_eb2c_4f59_8390_1afbbc8a793d.slice/crio-50bc1148e4864d5475a8d9eeb467ec542426d136fbedac182322689cf1146ce2 WatchSource:0}: Error finding container 50bc1148e4864d5475a8d9eeb467ec542426d136fbedac182322689cf1146ce2: Status 404 returned error can't find the container with id 50bc1148e4864d5475a8d9eeb467ec542426d136fbedac182322689cf1146ce2 Jul 07 12:49:46.698597 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.698515 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:49:46.722920 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.722886 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.726288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.726258 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 12:49:46.727603 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.727581 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 12:49:46.727693 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.727581 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 12:49:46.727693 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.727630 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 12:49:46.727693 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.727586 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 12:49:46.728673 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.728652 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 12:49:46.728794 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.728778 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ph9cx\"" Jul 07 12:49:46.728877 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.728862 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 12:49:46.729007 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.728992 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 12:49:46.729092 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.729055 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 12:49:46.745914 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.745882 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:49:46.827400 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827359 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827556 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827556 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827446 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfhzs\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-kube-api-access-zfhzs\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827556 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827476 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827556 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827511 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-web-config\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827556 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827533 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827560 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-volume\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827650 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-out\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827727 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.827812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827798 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.828099 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.827822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.915894 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.915838 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vcz6t" event={"ID":"9eb4ac60-a8e6-437d-b18b-631e3c984e70","Type":"ContainerStarted","Data":"61d7fbfe242359570654678aa7b69a8b62b751fc6c9666957fb08d4795d11b7e"} Jul 07 12:49:46.918852 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.918818 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" event={"ID":"a7790c18-eb2c-4f59-8390-1afbbc8a793d","Type":"ContainerStarted","Data":"dfee8554efde304cd98e88175711f5744993469304e13f286517a700e305eb33"} Jul 07 12:49:46.918958 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.918882 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" event={"ID":"a7790c18-eb2c-4f59-8390-1afbbc8a793d","Type":"ContainerStarted","Data":"1f33204a5064d1b6a94b86da343ef26c4eb5a0151f8669d5a7212d6ce024c253"} Jul 07 12:49:46.918958 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.918912 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" event={"ID":"a7790c18-eb2c-4f59-8390-1afbbc8a793d","Type":"ContainerStarted","Data":"50bc1148e4864d5475a8d9eeb467ec542426d136fbedac182322689cf1146ce2"} Jul 07 12:49:46.923181 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.923154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kzrz6" event={"ID":"d719beb6-22cd-4784-a0bb-463c9a856741","Type":"ContainerStarted","Data":"c20294d4a4fee0370bd767cc2a423d43b6075d0eef07d291d3b42c3d3c7be57c"} Jul 07 12:49:46.923857 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.923834 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kzrz6" event={"ID":"d719beb6-22cd-4784-a0bb-463c9a856741","Type":"ContainerStarted","Data":"bee9e772e219784efc587e65a168876ee2e4bdc85260d5eddef1e24b6472ef22"} Jul 07 12:49:46.928367 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.928309 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.928552 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.928536 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.928755 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.928737 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.928907 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.928891 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.928975 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.928948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929032 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.928987 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929032 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.929016 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfhzs\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-kube-api-access-zfhzs\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929146 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.929047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929146 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.929082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-web-config\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929146 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.929106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929289 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.929158 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-volume\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929289 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.929196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929289 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.929224 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-out\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.929955 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:46.929930 2568 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 12:49:46.930033 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:49:46.930005 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls podName:cbf91c8f-e9b2-457f-8680-fc0ab41e325a nodeName:}" failed. No retries permitted until 2026-07-07 12:49:47.429984691 +0000 UTC m=+69.414833155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a") : secret "alertmanager-main-tls" not found Jul 07 12:49:46.930173 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.930150 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.930998 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.930973 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.931707 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.931684 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.934049 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.934006 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.934049 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.934026 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.938401 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.936775 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-web-config\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.938401 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.937392 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.938401 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.937454 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-volume\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.938401 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.937780 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.938401 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.938013 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.938401 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.938152 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-out\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.941723 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.941691 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfhzs\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-kube-api-access-zfhzs\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:46.965623 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:46.965572 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kzrz6" podStartSLOduration=67.830667838 podStartE2EDuration="1m8.965553942s" totalImagesPullingTime="1.134886104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:46.964881858 +0000 UTC m=+68.949730322" watchObservedRunningTime="2026-07-07 12:49:46.965553942 +0000 UTC m=+68.950402415" Jul 07 12:49:47.434991 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:47.434896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:47.438002 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:47.437971 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:47.637298 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:47.637255 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:49:48.673046 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.673009 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-677f48795b-vpfkc"] Jul 07 12:49:48.676585 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.676564 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.682599 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.682577 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pg8jz\"" Jul 07 12:49:48.682781 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.682761 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2n6e7ks3rmdm8\"" Jul 07 12:49:48.683369 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.683351 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 12:49:48.683551 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.683383 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 12:49:48.683634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.683399 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 12:49:48.683693 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.683402 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 12:49:48.684854 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.684540 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 12:49:48.745996 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.745969 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-tls\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.746135 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.746008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.746135 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.746039 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.746135 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.746088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.746323 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.746132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-grpc-tls\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.746323 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.746158 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.746323 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.746177 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-metrics-client-ca\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.746323 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.746253 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h9qh\" (UniqueName: \"kubernetes.io/projected/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-kube-api-access-5h9qh\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.754646 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.754558 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-677f48795b-vpfkc"] Jul 07 12:49:48.847103 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.847068 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-tls\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.847290 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.847134 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.847290 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.847163 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.847290 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.847191 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.847290 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.847217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-grpc-tls\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.847290 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.847244 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.847290 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.847269 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-metrics-client-ca\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.847564 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.847325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h9qh\" (UniqueName: \"kubernetes.io/projected/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-kube-api-access-5h9qh\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.848309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.848257 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-metrics-client-ca\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.850233 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.850203 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.850345 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.850203 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.851038 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.850995 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.851174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.851134 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-grpc-tls\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.851174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.851155 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.854317 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.851505 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-secret-thanos-querier-tls\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.857881 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.857860 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h9qh\" (UniqueName: \"kubernetes.io/projected/38dbfefa-b283-4c6b-8b4e-34c26c3b3897-kube-api-access-5h9qh\") pod \"thanos-querier-677f48795b-vpfkc\" (UID: \"38dbfefa-b283-4c6b-8b4e-34c26c3b3897\") " pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:48.877042 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.877023 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w7t4f" Jul 07 12:49:48.988637 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:48.988602 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:50.226791 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.226730 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-677f48795b-vpfkc"] Jul 07 12:49:50.237314 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.237291 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:49:50.263142 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:50.263100 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dbfefa_b283_4c6b_8b4e_34c26c3b3897.slice/crio-7c0d6b23663b65552dfa2d12bb05e9ebf180db480a963deb4012ea9b4a68bfac WatchSource:0}: Error finding container 7c0d6b23663b65552dfa2d12bb05e9ebf180db480a963deb4012ea9b4a68bfac: Status 404 returned error can't find the container with id 7c0d6b23663b65552dfa2d12bb05e9ebf180db480a963deb4012ea9b4a68bfac Jul 07 12:49:50.263343 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:50.263325 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf91c8f_e9b2_457f_8680_fc0ab41e325a.slice/crio-09bdb5937c931321fd4f9260c76a83f81ec3d3765175c2eac8c254e3020cc6cd WatchSource:0}: Error finding container 09bdb5937c931321fd4f9260c76a83f81ec3d3765175c2eac8c254e3020cc6cd: Status 404 returned error can't find the container with id 09bdb5937c931321fd4f9260c76a83f81ec3d3765175c2eac8c254e3020cc6cd Jul 07 12:49:50.646100 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.646023 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86d4f56489-h4jcz"] Jul 07 12:49:50.941270 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.941191 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" event={"ID":"38dbfefa-b283-4c6b-8b4e-34c26c3b3897","Type":"ContainerStarted","Data":"7c0d6b23663b65552dfa2d12bb05e9ebf180db480a963deb4012ea9b4a68bfac"} Jul 07 12:49:50.943093 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.942874 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jqxc9" event={"ID":"275c0b51-3ae0-4606-8672-f047fa541200","Type":"ContainerStarted","Data":"00a01ee5f15b06128751649ac0ff4e7ace75898f5a43253b65dbd3f4897a873a"} Jul 07 12:49:50.944655 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.944625 2568 generic.go:356] "Generic (PLEG): container finished" podID="9eb4ac60-a8e6-437d-b18b-631e3c984e70" containerID="f4977ee8dd3f34e6271861ea60ee9dcf0310d96e7f566ae4939feb66d844d7ad" exitCode=0 Jul 07 12:49:50.944780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.944673 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vcz6t" event={"ID":"9eb4ac60-a8e6-437d-b18b-631e3c984e70","Type":"ContainerDied","Data":"f4977ee8dd3f34e6271861ea60ee9dcf0310d96e7f566ae4939feb66d844d7ad"} Jul 07 12:49:50.946361 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.946333 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerStarted","Data":"09bdb5937c931321fd4f9260c76a83f81ec3d3765175c2eac8c254e3020cc6cd"} Jul 07 12:49:50.948707 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.948284 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mv8d9" event={"ID":"5e34485f-5517-4209-ba1b-44ff7ebae506","Type":"ContainerStarted","Data":"181bce9eeee072d9a93bcc2bed129bf6a31ef94d9982e726361e8a3ac9aa669c"} Jul 07 12:49:50.948707 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.948506 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:49:50.951496 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.951471 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" event={"ID":"a7790c18-eb2c-4f59-8390-1afbbc8a793d","Type":"ContainerStarted","Data":"85b0f490c32834144d41a981121d69f6eaec5da1d7a396695ebc893e2189fe85"} Jul 07 12:49:50.981398 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:50.981106 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jqxc9" podStartSLOduration=67.667426571 podStartE2EDuration="1m12.98108658s" totalImagesPullingTime="5.313660009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:50.979923698 +0000 UTC m=+72.964772173" watchObservedRunningTime="2026-07-07 12:49:50.98108658 +0000 UTC m=+72.965935032" Jul 07 12:49:51.029419 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:51.029200 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sbxpf" podStartSLOduration=2.449778555 podStartE2EDuration="6.029181367s" totalImagesPullingTime="3.579402812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:51.027584421 +0000 UTC m=+73.012432896" watchObservedRunningTime="2026-07-07 12:49:51.029181367 +0000 UTC m=+73.014029841" Jul 07 12:49:51.056717 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:51.056676 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mv8d9" podStartSLOduration=67.7905854 podStartE2EDuration="1m13.056657682s" totalImagesPullingTime="5.266072282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:51.055574522 +0000 UTC m=+73.040423010" watchObservedRunningTime="2026-07-07 12:49:51.056657682 +0000 UTC m=+73.041506155" Jul 07 12:49:51.959093 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:51.958222 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vcz6t" event={"ID":"9eb4ac60-a8e6-437d-b18b-631e3c984e70","Type":"ContainerStarted","Data":"bdfcde720eea53493c6b528aba86b40d54c358d743ddb03a753ab4b4cf6b2f90"} Jul 07 12:49:51.959093 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:51.958333 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vcz6t" event={"ID":"9eb4ac60-a8e6-437d-b18b-631e3c984e70","Type":"ContainerStarted","Data":"140970d5210991900a553c9e8db035b73aba0f83a1d43f898548cc504c881536"} Jul 07 12:49:51.961225 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:51.961164 2568 generic.go:356] "Generic (PLEG): container finished" podID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerID="0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a" exitCode=0 Jul 07 12:49:51.961346 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:51.961260 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerDied","Data":"0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a"} Jul 07 12:49:51.983376 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:51.983031 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vcz6t" podStartSLOduration=2.9103623770000002 podStartE2EDuration="6.983014236s" totalImagesPullingTime="3.984854509s" totalInitContainerRuntime="87.79735ms" isStatefulPod=true podCreationTimestamp="2026-07-07 12:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:51.981666794 +0000 UTC m=+73.966515267" watchObservedRunningTime="2026-07-07 12:49:51.983014236 +0000 UTC m=+73.967862708" Jul 07 12:49:52.210739 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.210506 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:49:52.215431 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.213934 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.217019 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.216994 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 12:49:52.217669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.217618 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 12:49:52.217669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.217632 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 12:49:52.218395 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.218177 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4320ub25glgnn\"" Jul 07 12:49:52.218395 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.218185 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 12:49:52.219452 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.219426 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 12:49:52.219972 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.219926 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6cppc\"" Jul 07 12:49:52.220238 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.220156 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 12:49:52.220514 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.220429 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 12:49:52.221742 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.221706 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 12:49:52.222883 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.222579 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 12:49:52.223345 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.223179 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 12:49:52.245466 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.245443 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 12:49:52.246781 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.246753 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 12:49:52.247131 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.247093 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 12:49:52.247131 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.247130 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 12:49:52.260511 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.260488 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.281933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.281977 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282030 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282071 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282149 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282175 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282217 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjkr5\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-kube-api-access-hjkr5\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282248 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282292 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.282533 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282333 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-config\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.283161 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.283161 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.283161 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282406 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.283161 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.283161 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282468 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.283161 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282489 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.283161 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.282523 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383370 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383536 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383380 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383536 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383536 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383425 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383536 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383536 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjkr5\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-kube-api-access-hjkr5\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383799 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-config\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383894 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.383965 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.384724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383971 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.384724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.383996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.384724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.384022 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.384724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.384053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.384724 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.384364 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.384977 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.384899 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.386706 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.385949 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.386706 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.386393 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.386706 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.386509 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.386706 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.386634 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.387696 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.387322 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.387696 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.387398 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.387990 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.387966 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.388055 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.388027 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.388351 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.388303 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.388909 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.388866 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.388995 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.388961 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.389549 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.389509 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.389549 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.389522 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.390364 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.390324 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.390481 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.390456 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-config\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.390858 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.390834 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.391876 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.391854 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.399221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.399201 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjkr5\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-kube-api-access-hjkr5\") pod \"prometheus-k8s-0\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.405634 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.405510 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:49:52.528013 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.527970 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:49:52.887736 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.887681 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:49:52.890486 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:49:52.890461 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2fa50a8_958c_454b_911f_d9a50486a67c.slice/crio-c2e4ec7d2781f764feca5dd494dbb657fc45b2560b5577668ab170c453051d89 WatchSource:0}: Error finding container c2e4ec7d2781f764feca5dd494dbb657fc45b2560b5577668ab170c453051d89: Status 404 returned error can't find the container with id c2e4ec7d2781f764feca5dd494dbb657fc45b2560b5577668ab170c453051d89 Jul 07 12:49:52.970601 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.970543 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" event={"ID":"38dbfefa-b283-4c6b-8b4e-34c26c3b3897","Type":"ContainerStarted","Data":"3cd95122fe63fe5961463f6b0e705fdb3b0cd638e0512d4db08619bf9cc6fa2b"} Jul 07 12:49:52.970946 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.970623 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" event={"ID":"38dbfefa-b283-4c6b-8b4e-34c26c3b3897","Type":"ContainerStarted","Data":"9914383a4ec8b3732a47eeaef9d61e12e82c7ce62d4e6536f0068b7853eca762"} Jul 07 12:49:52.970946 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.970673 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" event={"ID":"38dbfefa-b283-4c6b-8b4e-34c26c3b3897","Type":"ContainerStarted","Data":"e64aede28198712017f03e38529fb3ad21325121b057d8969b39cc3834d447e6"} Jul 07 12:49:52.975019 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.974936 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerStarted","Data":"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9"} Jul 07 12:49:52.975019 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:52.975005 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerStarted","Data":"c2e4ec7d2781f764feca5dd494dbb657fc45b2560b5577668ab170c453051d89"} Jul 07 12:49:53.981453 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:53.981389 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" exitCode=0 Jul 07 12:49:53.982000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:53.981939 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerDied","Data":"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9"} Jul 07 12:49:54.990497 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.990454 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" event={"ID":"38dbfefa-b283-4c6b-8b4e-34c26c3b3897","Type":"ContainerStarted","Data":"40ef0b21882ea364a9d8eb405cbd65ed298c1a9f2ed3a0077a4b53da71346f04"} Jul 07 12:49:54.990972 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.990578 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" event={"ID":"38dbfefa-b283-4c6b-8b4e-34c26c3b3897","Type":"ContainerStarted","Data":"c4311e65bbae4a5e777f3e1db81cbfc71691f38a71886a2f2f5e15e1c4ecec6b"} Jul 07 12:49:54.990972 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.990625 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:49:54.990972 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.990664 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" event={"ID":"38dbfefa-b283-4c6b-8b4e-34c26c3b3897","Type":"ContainerStarted","Data":"526c58e600438bc6fa6b3fab5f727dfea2d15a8ba5e628fbd0cfae2c97ae1e91"} Jul 07 12:49:54.994319 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.994286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerStarted","Data":"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af"} Jul 07 12:49:54.994431 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.994358 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerStarted","Data":"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7"} Jul 07 12:49:54.994431 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.994399 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerStarted","Data":"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d"} Jul 07 12:49:54.994547 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.994444 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerStarted","Data":"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c"} Jul 07 12:49:54.994547 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.994485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerStarted","Data":"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f"} Jul 07 12:49:54.994547 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:54.994525 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerStarted","Data":"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42"} Jul 07 12:49:55.036022 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:55.035967 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" podStartSLOduration=3.521883795 podStartE2EDuration="7.035948733s" totalImagesPullingTime="3.514064938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:55.033433566 +0000 UTC m=+77.018282067" watchObservedRunningTime="2026-07-07 12:49:55.035948733 +0000 UTC m=+77.020797208" Jul 07 12:49:55.077918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:55.077860 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.9121165730000005 podStartE2EDuration="9.077837393s" totalImagesPullingTime="3.150876342s" totalInitContainerRuntime="14.844478ms" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:55.076955909 +0000 UTC m=+77.061804386" watchObservedRunningTime="2026-07-07 12:49:55.077837393 +0000 UTC m=+77.062685866" Jul 07 12:49:58.012697 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:58.012667 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerStarted","Data":"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f"} Jul 07 12:49:58.013053 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:58.012752 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerStarted","Data":"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c"} Jul 07 12:49:58.013053 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:58.012784 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerStarted","Data":"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb"} Jul 07 12:49:58.013053 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:58.012809 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerStarted","Data":"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7"} Jul 07 12:49:58.013053 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:58.012849 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerStarted","Data":"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f"} Jul 07 12:49:58.013053 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:58.012875 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerStarted","Data":"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7"} Jul 07 12:49:58.059568 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:49:58.059512 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.715198855 podStartE2EDuration="6.059492844s" totalImagesPullingTime="3.317195071s" totalInitContainerRuntime="27.098918ms" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:58.057335913 +0000 UTC m=+80.042184406" watchObservedRunningTime="2026-07-07 12:49:58.059492844 +0000 UTC m=+80.044341319" Jul 07 12:50:01.007050 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:01.006944 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-677f48795b-vpfkc" Jul 07 12:50:02.528436 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:02.528402 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:50:15.665434 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:15.665382 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-86d4f56489-h4jcz" podUID="1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" containerName="console" containerID="cri-o://793f8adf71135e70159b76700667128041f0ab6311b297a719d9b0bc457f7519" gracePeriod=15 Jul 07 12:50:16.083045 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.083013 2568 generic.go:356] "Generic (PLEG): container finished" podID="1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" containerID="793f8adf71135e70159b76700667128041f0ab6311b297a719d9b0bc457f7519" exitCode=0 Jul 07 12:50:16.083239 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.083091 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d4f56489-h4jcz" event={"ID":"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5","Type":"ContainerDied","Data":"793f8adf71135e70159b76700667128041f0ab6311b297a719d9b0bc457f7519"} Jul 07 12:50:16.403747 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.403722 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:50:16.499397 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499362 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-serving-cert\") pod \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " Jul 07 12:50:16.499565 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499419 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-oauth-serving-cert\") pod \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " Jul 07 12:50:16.499565 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499446 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-service-ca\") pod \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " Jul 07 12:50:16.499565 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499470 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-config\") pod \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " Jul 07 12:50:16.499565 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499512 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-oauth-config\") pod \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " Jul 07 12:50:16.499764 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499573 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thjhr\" (UniqueName: \"kubernetes.io/projected/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-kube-api-access-thjhr\") pod \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " Jul 07 12:50:16.499764 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499626 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-trusted-ca-bundle\") pod \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\" (UID: \"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5\") " Jul 07 12:50:16.499859 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499771 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" (UID: "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:50:16.499859 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499823 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-service-ca" (OuterVolumeSpecName: "service-ca") pod "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" (UID: "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:50:16.500001 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.499915 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-oauth-serving-cert\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:50:16.500062 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.500010 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:50:16.500109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.500054 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-config" (OuterVolumeSpecName: "console-config") pod "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" (UID: "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:50:16.500109 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.500077 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" (UID: "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:50:16.501730 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.501705 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" (UID: "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:50:16.501822 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.501747 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-kube-api-access-thjhr" (OuterVolumeSpecName: "kube-api-access-thjhr") pod "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" (UID: "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5"). InnerVolumeSpecName "kube-api-access-thjhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:50:16.501822 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.501771 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" (UID: "1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:50:16.601091 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.601062 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-oauth-config\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:50:16.601091 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.601092 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thjhr\" (UniqueName: \"kubernetes.io/projected/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-kube-api-access-thjhr\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:50:16.601277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.601107 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-trusted-ca-bundle\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:50:16.601277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.601138 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-serving-cert\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:50:16.601277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:16.601151 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5-console-config\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:50:17.088600 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:17.088571 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d4f56489-h4jcz" Jul 07 12:50:17.088997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:17.088570 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d4f56489-h4jcz" event={"ID":"1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5","Type":"ContainerDied","Data":"92e230a7049903e72a4f184d678702f5dff6c8b1c7e4292f37886a6a006fcd88"} Jul 07 12:50:17.088997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:17.088719 2568 scope.go:117] "RemoveContainer" containerID="793f8adf71135e70159b76700667128041f0ab6311b297a719d9b0bc457f7519" Jul 07 12:50:17.121497 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:17.121472 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86d4f56489-h4jcz"] Jul 07 12:50:17.124547 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:17.124527 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86d4f56489-h4jcz"] Jul 07 12:50:18.602239 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:18.602207 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" path="/var/lib/kubelet/pods/1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5/volumes" Jul 07 12:50:21.963667 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:21.963638 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mv8d9" Jul 07 12:50:29.225493 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:29.225454 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5hx9w_dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9/serve-healthcheck-canary/0.log" Jul 07 12:50:52.529047 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:52.529001 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:50:52.549050 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:52.549025 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:50:53.241487 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:50:53.241458 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:06.199277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:06.199241 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:51:06.199827 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:06.199652 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="alertmanager" containerID="cri-o://256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42" gracePeriod=120 Jul 07 12:51:06.199827 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:06.199719 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy-metric" containerID="cri-o://b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7" gracePeriod=120 Jul 07 12:51:06.199827 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:06.199778 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="prom-label-proxy" containerID="cri-o://bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af" gracePeriod=120 Jul 07 12:51:06.199827 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:06.199781 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="config-reloader" containerID="cri-o://2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f" gracePeriod=120 Jul 07 12:51:06.200029 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:06.199880 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy-web" containerID="cri-o://bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c" gracePeriod=120 Jul 07 12:51:06.200029 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:06.199781 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy" containerID="cri-o://c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d" gracePeriod=120 Jul 07 12:51:07.280430 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.280399 2568 generic.go:356] "Generic (PLEG): container finished" podID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerID="bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af" exitCode=0 Jul 07 12:51:07.280430 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.280428 2568 generic.go:356] "Generic (PLEG): container finished" podID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerID="c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d" exitCode=0 Jul 07 12:51:07.280731 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.280438 2568 generic.go:356] "Generic (PLEG): container finished" podID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerID="2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f" exitCode=0 Jul 07 12:51:07.280731 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.280447 2568 generic.go:356] "Generic (PLEG): container finished" podID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerID="256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42" exitCode=0 Jul 07 12:51:07.280731 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.280459 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerDied","Data":"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af"} Jul 07 12:51:07.280731 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.280523 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerDied","Data":"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d"} Jul 07 12:51:07.280731 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.280550 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerDied","Data":"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f"} Jul 07 12:51:07.280731 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.280575 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerDied","Data":"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42"} Jul 07 12:51:07.447430 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.446898 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:07.471930 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.471906 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-web\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472078 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.471956 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472078 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.471983 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-trusted-ca-bundle\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472229 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472095 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-cluster-tls-config\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472229 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472156 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-tls-assets\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472229 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472203 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfhzs\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-kube-api-access-zfhzs\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472229 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472225 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472423 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472251 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-main-db\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472423 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472287 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472423 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472332 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-volume\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472423 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472355 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:07.472423 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472373 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-web-config\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472423 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472403 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-metrics-client-ca\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472708 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472438 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-out\") pod \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\" (UID: \"cbf91c8f-e9b2-457f-8680-fc0ab41e325a\") " Jul 07 12:51:07.472708 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472608 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:51:07.472822 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472729 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.472822 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.472749 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-alertmanager-main-db\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.473455 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.473424 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:07.475471 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.475442 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:07.475571 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.475515 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:07.475632 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.475599 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:51:07.475781 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.475759 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-out" (OuterVolumeSpecName: "config-out") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:51:07.475841 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.475770 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:07.476467 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.476435 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-kube-api-access-zfhzs" (OuterVolumeSpecName: "kube-api-access-zfhzs") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "kube-api-access-zfhzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:51:07.476560 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.476483 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-volume" (OuterVolumeSpecName: "config-volume") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:07.476701 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.476685 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:07.479408 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.479386 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:07.485974 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.485954 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-web-config" (OuterVolumeSpecName: "web-config") pod "cbf91c8f-e9b2-457f-8680-fc0ab41e325a" (UID: "cbf91c8f-e9b2-457f-8680-fc0ab41e325a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:07.573436 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573361 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573436 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573391 2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-volume\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573436 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573403 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-web-config\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573436 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573411 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-metrics-client-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573436 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573419 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-config-out\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573436 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573428 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573436 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573438 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-main-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573738 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573446 2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-cluster-tls-config\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573738 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573459 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-tls-assets\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573738 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573472 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfhzs\" (UniqueName: \"kubernetes.io/projected/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-kube-api-access-zfhzs\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:07.573738 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:07.573482 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cbf91c8f-e9b2-457f-8680-fc0ab41e325a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:08.286940 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.286903 2568 generic.go:356] "Generic (PLEG): container finished" podID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerID="b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7" exitCode=0 Jul 07 12:51:08.286940 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.286930 2568 generic.go:356] "Generic (PLEG): container finished" podID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerID="bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c" exitCode=0 Jul 07 12:51:08.287406 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.286972 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerDied","Data":"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7"} Jul 07 12:51:08.287406 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.286990 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.287406 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.287031 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerDied","Data":"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c"} Jul 07 12:51:08.287406 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.287058 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cbf91c8f-e9b2-457f-8680-fc0ab41e325a","Type":"ContainerDied","Data":"09bdb5937c931321fd4f9260c76a83f81ec3d3765175c2eac8c254e3020cc6cd"} Jul 07 12:51:08.287406 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.287086 2568 scope.go:117] "RemoveContainer" containerID="bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af" Jul 07 12:51:08.295573 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.295553 2568 scope.go:117] "RemoveContainer" containerID="b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7" Jul 07 12:51:08.302084 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.302066 2568 scope.go:117] "RemoveContainer" containerID="c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d" Jul 07 12:51:08.308684 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.308665 2568 scope.go:117] "RemoveContainer" containerID="bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c" Jul 07 12:51:08.315033 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.314855 2568 scope.go:117] "RemoveContainer" containerID="2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f" Jul 07 12:51:08.321772 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.321757 2568 scope.go:117] "RemoveContainer" containerID="256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42" Jul 07 12:51:08.324154 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.324135 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:51:08.328173 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.328149 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:51:08.329815 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.329800 2568 scope.go:117] "RemoveContainer" containerID="0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a" Jul 07 12:51:08.336377 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.336353 2568 scope.go:117] "RemoveContainer" containerID="bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af" Jul 07 12:51:08.336612 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:08.336591 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af\": container with ID starting with bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af not found: ID does not exist" containerID="bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af" Jul 07 12:51:08.336656 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.336622 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af"} err="failed to get container status \"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af\": rpc error: code = NotFound desc = could not find container \"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af\": container with ID starting with bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af not found: ID does not exist" Jul 07 12:51:08.336656 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.336655 2568 scope.go:117] "RemoveContainer" containerID="b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7" Jul 07 12:51:08.336873 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:08.336856 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7\": container with ID starting with b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7 not found: ID does not exist" containerID="b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7" Jul 07 12:51:08.336914 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.336880 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7"} err="failed to get container status \"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7\": rpc error: code = NotFound desc = could not find container \"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7\": container with ID starting with b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7 not found: ID does not exist" Jul 07 12:51:08.336914 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.336895 2568 scope.go:117] "RemoveContainer" containerID="c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d" Jul 07 12:51:08.337103 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:08.337090 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d\": container with ID starting with c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d not found: ID does not exist" containerID="c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d" Jul 07 12:51:08.337170 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.337107 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d"} err="failed to get container status \"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d\": rpc error: code = NotFound desc = could not find container \"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d\": container with ID starting with c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d not found: ID does not exist" Jul 07 12:51:08.337170 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.337156 2568 scope.go:117] "RemoveContainer" containerID="bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c" Jul 07 12:51:08.337384 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:08.337367 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c\": container with ID starting with bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c not found: ID does not exist" containerID="bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c" Jul 07 12:51:08.337425 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.337390 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c"} err="failed to get container status \"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c\": rpc error: code = NotFound desc = could not find container \"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c\": container with ID starting with bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c not found: ID does not exist" Jul 07 12:51:08.337425 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.337405 2568 scope.go:117] "RemoveContainer" containerID="2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f" Jul 07 12:51:08.337631 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:08.337614 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f\": container with ID starting with 2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f not found: ID does not exist" containerID="2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f" Jul 07 12:51:08.337673 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.337636 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f"} err="failed to get container status \"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f\": rpc error: code = NotFound desc = could not find container \"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f\": container with ID starting with 2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f not found: ID does not exist" Jul 07 12:51:08.337673 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.337648 2568 scope.go:117] "RemoveContainer" containerID="256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42" Jul 07 12:51:08.337865 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:08.337847 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42\": container with ID starting with 256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42 not found: ID does not exist" containerID="256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42" Jul 07 12:51:08.337925 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.337871 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42"} err="failed to get container status \"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42\": rpc error: code = NotFound desc = could not find container \"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42\": container with ID starting with 256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42 not found: ID does not exist" Jul 07 12:51:08.337925 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.337891 2568 scope.go:117] "RemoveContainer" containerID="0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a" Jul 07 12:51:08.338150 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:08.338134 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a\": container with ID starting with 0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a not found: ID does not exist" containerID="0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a" Jul 07 12:51:08.338208 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338153 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a"} err="failed to get container status \"0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a\": rpc error: code = NotFound desc = could not find container \"0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a\": container with ID starting with 0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a not found: ID does not exist" Jul 07 12:51:08.338208 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338165 2568 scope.go:117] "RemoveContainer" containerID="bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af" Jul 07 12:51:08.338353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338336 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af"} err="failed to get container status \"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af\": rpc error: code = NotFound desc = could not find container \"bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af\": container with ID starting with bba0976a593ec4bd7cd9cb73b4ac33ec244d82d09291649b430d997f13fbf9af not found: ID does not exist" Jul 07 12:51:08.338405 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338354 2568 scope.go:117] "RemoveContainer" containerID="b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7" Jul 07 12:51:08.338591 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338574 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7"} err="failed to get container status \"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7\": rpc error: code = NotFound desc = could not find container \"b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7\": container with ID starting with b75bd6e03dae6c17e93b91a9b9f35ea040ea7987efa80506381f1f945811c7d7 not found: ID does not exist" Jul 07 12:51:08.338662 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338596 2568 scope.go:117] "RemoveContainer" containerID="c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d" Jul 07 12:51:08.338825 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338808 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d"} err="failed to get container status \"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d\": rpc error: code = NotFound desc = could not find container \"c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d\": container with ID starting with c25d67a0fadae8be01a00761831d376d31a08b2ecab3ef23a1d0060e454b6c8d not found: ID does not exist" Jul 07 12:51:08.338872 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338825 2568 scope.go:117] "RemoveContainer" containerID="bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c" Jul 07 12:51:08.338986 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338973 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c"} err="failed to get container status \"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c\": rpc error: code = NotFound desc = could not find container \"bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c\": container with ID starting with bf98364c4d08a811a4e7890ef09e728732e0d2642604c06024e3f6787fc4bf5c not found: ID does not exist" Jul 07 12:51:08.339024 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.338987 2568 scope.go:117] "RemoveContainer" containerID="2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f" Jul 07 12:51:08.339200 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.339182 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f"} err="failed to get container status \"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f\": rpc error: code = NotFound desc = could not find container \"2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f\": container with ID starting with 2539449ee76f56573bf8c21d129104868d4e640c258b164764f139788dc9644f not found: ID does not exist" Jul 07 12:51:08.339279 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.339201 2568 scope.go:117] "RemoveContainer" containerID="256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42" Jul 07 12:51:08.339435 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.339416 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42"} err="failed to get container status \"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42\": rpc error: code = NotFound desc = could not find container \"256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42\": container with ID starting with 256c680342dfb84a0659c1acc8ed5ae8df3b9ba8caf0167b758d864cb3211c42 not found: ID does not exist" Jul 07 12:51:08.339475 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.339436 2568 scope.go:117] "RemoveContainer" containerID="0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a" Jul 07 12:51:08.339621 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.339605 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a"} err="failed to get container status \"0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a\": rpc error: code = NotFound desc = could not find container \"0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a\": container with ID starting with 0a9c2736ed4166022f408740a43b05b2d8c96540d84933d81ff66fb12ac6490a not found: ID does not exist" Jul 07 12:51:08.376478 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376457 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:51:08.376713 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376702 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" containerName="console" Jul 07 12:51:08.376754 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376714 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" containerName="console" Jul 07 12:51:08.376754 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376725 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy-web" Jul 07 12:51:08.376754 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376730 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy-web" Jul 07 12:51:08.376754 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376741 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="config-reloader" Jul 07 12:51:08.376754 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376746 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="config-reloader" Jul 07 12:51:08.376754 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376752 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy-metric" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376758 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy-metric" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376766 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="alertmanager" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376771 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="alertmanager" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376778 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="prom-label-proxy" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376783 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="prom-label-proxy" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376789 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376794 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376801 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="init-config-reloader" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376805 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="init-config-reloader" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376849 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8de0b7-7e1d-45b7-80ac-e46734bb1ab5" containerName="console" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376855 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="alertmanager" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376860 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy-metric" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376866 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="config-reloader" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376871 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy-web" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376876 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="prom-label-proxy" Jul 07 12:51:08.376918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.376880 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" containerName="kube-rbac-proxy" Jul 07 12:51:08.381130 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.381102 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.399395 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.399370 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 12:51:08.399585 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.399571 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 12:51:08.400226 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.400213 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 12:51:08.400274 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.400245 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 12:51:08.400531 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.400519 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 12:51:08.400619 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.400601 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 12:51:08.400927 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.400912 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ph9cx\"" Jul 07 12:51:08.400967 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.400916 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 12:51:08.409003 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.408989 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 12:51:08.417036 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.417021 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 12:51:08.481024 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.480993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481024 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481026 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d08c2688-4a25-4a4a-87c4-256502309af1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481274 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481051 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d08c2688-4a25-4a4a-87c4-256502309af1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481274 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481274 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4b6k\" (UniqueName: \"kubernetes.io/projected/d08c2688-4a25-4a4a-87c4-256502309af1-kube-api-access-m4b6k\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481385 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481292 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481385 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481336 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-config-volume\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481385 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481385 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481374 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481535 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481407 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d08c2688-4a25-4a4a-87c4-256502309af1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481535 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d08c2688-4a25-4a4a-87c4-256502309af1-config-out\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481535 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481449 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-web-config\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.481535 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.481481 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d08c2688-4a25-4a4a-87c4-256502309af1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.527508 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.527420 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:51:08.581877 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.581851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-config-volume\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.581877 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.581880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582142 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.581902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582142 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582018 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d08c2688-4a25-4a4a-87c4-256502309af1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582142 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582051 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d08c2688-4a25-4a4a-87c4-256502309af1-config-out\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582142 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582068 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-web-config\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582341 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d08c2688-4a25-4a4a-87c4-256502309af1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582341 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582341 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582313 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d08c2688-4a25-4a4a-87c4-256502309af1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582508 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d08c2688-4a25-4a4a-87c4-256502309af1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582508 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582508 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4b6k\" (UniqueName: \"kubernetes.io/projected/d08c2688-4a25-4a4a-87c4-256502309af1-kube-api-access-m4b6k\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582508 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582463 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.582508 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.582465 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d08c2688-4a25-4a4a-87c4-256502309af1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.583178 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.583150 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d08c2688-4a25-4a4a-87c4-256502309af1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.584563 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.584535 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d08c2688-4a25-4a4a-87c4-256502309af1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.585090 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.585034 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d08c2688-4a25-4a4a-87c4-256502309af1-config-out\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.585211 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.585152 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.585211 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.585154 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.585379 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.585341 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-config-volume\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.585814 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.585788 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.585918 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.585889 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.586231 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.586213 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.586623 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.586607 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d08c2688-4a25-4a4a-87c4-256502309af1-web-config\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.586811 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.586793 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d08c2688-4a25-4a4a-87c4-256502309af1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.593729 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.593707 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4b6k\" (UniqueName: \"kubernetes.io/projected/d08c2688-4a25-4a4a-87c4-256502309af1-kube-api-access-m4b6k\") pod \"alertmanager-main-0\" (UID: \"d08c2688-4a25-4a4a-87c4-256502309af1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:08.602941 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.602920 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbf91c8f-e9b2-457f-8680-fc0ab41e325a" path="/var/lib/kubelet/pods/cbf91c8f-e9b2-457f-8680-fc0ab41e325a/volumes" Jul 07 12:51:08.689174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:08.689148 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:51:09.022952 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:09.022930 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:51:09.024138 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:51:09.024097 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08c2688_4a25_4a4a_87c4_256502309af1.slice/crio-8dd5282aef3366dc30d7990b41f17f38a3bc8aa69f0c4f807b7f40ce774a1e3d WatchSource:0}: Error finding container 8dd5282aef3366dc30d7990b41f17f38a3bc8aa69f0c4f807b7f40ce774a1e3d: Status 404 returned error can't find the container with id 8dd5282aef3366dc30d7990b41f17f38a3bc8aa69f0c4f807b7f40ce774a1e3d Jul 07 12:51:09.294158 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:09.294052 2568 generic.go:356] "Generic (PLEG): container finished" podID="d08c2688-4a25-4a4a-87c4-256502309af1" containerID="e4eab5530a9018b3cd496f5cf4b9cee814ebf566811f2c96397f90d1da183a28" exitCode=0 Jul 07 12:51:09.294573 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:09.294151 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d08c2688-4a25-4a4a-87c4-256502309af1","Type":"ContainerDied","Data":"e4eab5530a9018b3cd496f5cf4b9cee814ebf566811f2c96397f90d1da183a28"} Jul 07 12:51:09.294573 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:09.294236 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d08c2688-4a25-4a4a-87c4-256502309af1","Type":"ContainerStarted","Data":"8dd5282aef3366dc30d7990b41f17f38a3bc8aa69f0c4f807b7f40ce774a1e3d"} Jul 07 12:51:10.306830 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.306796 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d08c2688-4a25-4a4a-87c4-256502309af1","Type":"ContainerStarted","Data":"536d5fbe29a1ba2cadd04ca22da73074445bc0f64824ed39f2575c1df9f87973"} Jul 07 12:51:10.307288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.306878 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d08c2688-4a25-4a4a-87c4-256502309af1","Type":"ContainerStarted","Data":"8094c727187c6e6d2cacc8320f090edca1aad6e4281662fbec3e08a55e8b5b53"} Jul 07 12:51:10.307288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.306906 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d08c2688-4a25-4a4a-87c4-256502309af1","Type":"ContainerStarted","Data":"29c3b683274bcc9d9b017d1598d045f5e5c2311dc7492deabcea7ac729353a8c"} Jul 07 12:51:10.307288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.306929 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d08c2688-4a25-4a4a-87c4-256502309af1","Type":"ContainerStarted","Data":"99648c47177d8cd0131006e63ff527dfa936d342f437f73fa307b641572c42a4"} Jul 07 12:51:10.307288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.306951 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d08c2688-4a25-4a4a-87c4-256502309af1","Type":"ContainerStarted","Data":"f23bab7555a3400166b12304501e425f1598d7dd333bb07ea63d356d620b391f"} Jul 07 12:51:10.307288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.306973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d08c2688-4a25-4a4a-87c4-256502309af1","Type":"ContainerStarted","Data":"bb32a67c04d9d56a2522b0dee6faa8cab2c4cd8a5c862df1b49960f1061a475f"} Jul 07 12:51:10.344496 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.344445 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.324338972 podStartE2EDuration="2.344430153s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.091181ms" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:10.343218292 +0000 UTC m=+152.328066765" watchObservedRunningTime="2026-07-07 12:51:10.344430153 +0000 UTC m=+152.329278624" Jul 07 12:51:10.588178 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.588087 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:51:10.588481 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.588447 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="prometheus" containerID="cri-o://d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" gracePeriod=600 Jul 07 12:51:10.589056 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.588656 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy" containerID="cri-o://bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" gracePeriod=600 Jul 07 12:51:10.590106 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.588827 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" gracePeriod=600 Jul 07 12:51:10.590412 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.588845 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="thanos-sidecar" containerID="cri-o://21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" gracePeriod=600 Jul 07 12:51:10.590412 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.588860 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy-web" containerID="cri-o://d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" gracePeriod=600 Jul 07 12:51:10.590412 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.588881 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="config-reloader" containerID="cri-o://d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" gracePeriod=600 Jul 07 12:51:10.848527 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.848494 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:10.904342 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904303 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-1\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904342 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904352 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-config-out\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904382 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904407 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-kube-rbac-proxy\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904434 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-trusted-ca-bundle\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904449 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-kubelet-serving-ca-bundle\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904485 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjkr5\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-kube-api-access-hjkr5\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904501 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-metrics-client-certs\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904531 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-config\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904559 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-db\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904580 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-tls\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.904607 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904606 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.905104 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904636 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-grpc-tls\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.905104 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904668 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-metrics-client-ca\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.905104 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904683 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:10.905104 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904707 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-2\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.905104 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904744 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-serving-certs-ca-bundle\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.905104 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904768 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-web-config\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.905104 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904796 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-thanos-prometheus-http-client-file\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.905104 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.904820 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-tls-assets\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.905512 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.905153 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:10.905512 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.905241 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:10.905617 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.905522 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.905833 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.906067 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.906392 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-0\") pod \"a2fa50a8-958c-454b-911f-d9a50486a67c\" (UID: \"a2fa50a8-958c-454b-911f-d9a50486a67c\") " Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.906996 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-config-out" (OuterVolumeSpecName: "config-out") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.907037 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-metrics-client-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.907058 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.907073 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.907088 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.907102 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:10.907193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.907134 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:10.907771 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.907596 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:51:10.908036 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.907987 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:10.908515 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.908476 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-config" (OuterVolumeSpecName: "config") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:10.909570 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.909541 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:10.909669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.909586 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:51:10.909669 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.909633 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:10.909780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.909692 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:10.909780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.909713 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:10.909780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.909742 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-kube-api-access-hjkr5" (OuterVolumeSpecName: "kube-api-access-hjkr5") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "kube-api-access-hjkr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:51:10.910267 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.910245 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:10.910901 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.910868 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:10.911131 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.911086 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:10.919600 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:10.919574 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-web-config" (OuterVolumeSpecName: "web-config") pod "a2fa50a8-958c-454b-911f-d9a50486a67c" (UID: "a2fa50a8-958c-454b-911f-d9a50486a67c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:51:11.008070 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008037 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-config-out\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008070 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008063 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008070 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008073 2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-kube-rbac-proxy\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008083 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjkr5\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-kube-api-access-hjkr5\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008092 2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-metrics-client-certs\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008100 2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-config\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008108 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-db\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008138 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008147 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008156 2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-secret-grpc-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008189 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-web-config\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008198 2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2fa50a8-958c-454b-911f-d9a50486a67c-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008209 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2fa50a8-958c-454b-911f-d9a50486a67c-tls-assets\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.008309 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.008217 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2fa50a8-958c-454b-911f-d9a50486a67c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 12:51:11.314854 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314811 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" exitCode=0 Jul 07 12:51:11.314854 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314841 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" exitCode=0 Jul 07 12:51:11.314854 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314853 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" exitCode=0 Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314867 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" exitCode=0 Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314876 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" exitCode=0 Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314882 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" exitCode=0 Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314888 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314849 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerDied","Data":"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f"} Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.314994 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerDied","Data":"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c"} Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.315030 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerDied","Data":"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb"} Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.315063 2568 scope.go:117] "RemoveContainer" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.315072 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerDied","Data":"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7"} Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.315124 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerDied","Data":"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f"} Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.315165 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerDied","Data":"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7"} Jul 07 12:51:11.315348 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.315204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2fa50a8-958c-454b-911f-d9a50486a67c","Type":"ContainerDied","Data":"c2e4ec7d2781f764feca5dd494dbb657fc45b2560b5577668ab170c453051d89"} Jul 07 12:51:11.323131 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.323074 2568 scope.go:117] "RemoveContainer" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" Jul 07 12:51:11.329773 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.329754 2568 scope.go:117] "RemoveContainer" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" Jul 07 12:51:11.336088 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.336072 2568 scope.go:117] "RemoveContainer" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" Jul 07 12:51:11.342135 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.342099 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:51:11.342960 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.342943 2568 scope.go:117] "RemoveContainer" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" Jul 07 12:51:11.346913 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.346895 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:51:11.349834 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.349818 2568 scope.go:117] "RemoveContainer" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" Jul 07 12:51:11.356451 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.356437 2568 scope.go:117] "RemoveContainer" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" Jul 07 12:51:11.362517 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.362502 2568 scope.go:117] "RemoveContainer" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" Jul 07 12:51:11.362736 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:11.362715 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": container with ID starting with 1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f not found: ID does not exist" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" Jul 07 12:51:11.362787 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.362744 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f"} err="failed to get container status \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": rpc error: code = NotFound desc = could not find container \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": container with ID starting with 1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f not found: ID does not exist" Jul 07 12:51:11.362787 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.362763 2568 scope.go:117] "RemoveContainer" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" Jul 07 12:51:11.362989 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:11.362972 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": container with ID starting with bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c not found: ID does not exist" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" Jul 07 12:51:11.363024 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.362994 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c"} err="failed to get container status \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": rpc error: code = NotFound desc = could not find container \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": container with ID starting with bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c not found: ID does not exist" Jul 07 12:51:11.363024 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363007 2568 scope.go:117] "RemoveContainer" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" Jul 07 12:51:11.363237 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:11.363220 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": container with ID starting with d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb not found: ID does not exist" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" Jul 07 12:51:11.363281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363240 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb"} err="failed to get container status \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": rpc error: code = NotFound desc = could not find container \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": container with ID starting with d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb not found: ID does not exist" Jul 07 12:51:11.363281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363253 2568 scope.go:117] "RemoveContainer" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" Jul 07 12:51:11.363493 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:11.363474 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": container with ID starting with 21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7 not found: ID does not exist" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" Jul 07 12:51:11.363543 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363496 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7"} err="failed to get container status \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": rpc error: code = NotFound desc = could not find container \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": container with ID starting with 21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7 not found: ID does not exist" Jul 07 12:51:11.363543 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363508 2568 scope.go:117] "RemoveContainer" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" Jul 07 12:51:11.363692 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:11.363676 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": container with ID starting with d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f not found: ID does not exist" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" Jul 07 12:51:11.363729 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363695 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f"} err="failed to get container status \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": rpc error: code = NotFound desc = could not find container \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": container with ID starting with d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f not found: ID does not exist" Jul 07 12:51:11.363729 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363707 2568 scope.go:117] "RemoveContainer" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" Jul 07 12:51:11.363886 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:11.363869 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": container with ID starting with d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7 not found: ID does not exist" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" Jul 07 12:51:11.363926 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363889 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7"} err="failed to get container status \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": rpc error: code = NotFound desc = could not find container \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": container with ID starting with d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7 not found: ID does not exist" Jul 07 12:51:11.363926 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.363900 2568 scope.go:117] "RemoveContainer" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" Jul 07 12:51:11.364107 ip-10-0-131-17 kubenswrapper[2568]: E0707 12:51:11.364091 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": container with ID starting with 3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9 not found: ID does not exist" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" Jul 07 12:51:11.364199 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364108 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9"} err="failed to get container status \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": rpc error: code = NotFound desc = could not find container \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": container with ID starting with 3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9 not found: ID does not exist" Jul 07 12:51:11.364199 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364141 2568 scope.go:117] "RemoveContainer" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" Jul 07 12:51:11.364316 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364300 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f"} err="failed to get container status \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": rpc error: code = NotFound desc = could not find container \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": container with ID starting with 1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f not found: ID does not exist" Jul 07 12:51:11.364367 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364316 2568 scope.go:117] "RemoveContainer" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" Jul 07 12:51:11.364527 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364502 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c"} err="failed to get container status \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": rpc error: code = NotFound desc = could not find container \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": container with ID starting with bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c not found: ID does not exist" Jul 07 12:51:11.364588 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364532 2568 scope.go:117] "RemoveContainer" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" Jul 07 12:51:11.364786 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364769 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb"} err="failed to get container status \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": rpc error: code = NotFound desc = could not find container \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": container with ID starting with d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb not found: ID does not exist" Jul 07 12:51:11.364834 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364788 2568 scope.go:117] "RemoveContainer" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" Jul 07 12:51:11.364991 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364976 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7"} err="failed to get container status \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": rpc error: code = NotFound desc = could not find container \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": container with ID starting with 21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7 not found: ID does not exist" Jul 07 12:51:11.365029 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.364990 2568 scope.go:117] "RemoveContainer" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" Jul 07 12:51:11.365269 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.365248 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f"} err="failed to get container status \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": rpc error: code = NotFound desc = could not find container \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": container with ID starting with d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f not found: ID does not exist" Jul 07 12:51:11.365346 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.365270 2568 scope.go:117] "RemoveContainer" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" Jul 07 12:51:11.365539 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.365521 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7"} err="failed to get container status \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": rpc error: code = NotFound desc = could not find container \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": container with ID starting with d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7 not found: ID does not exist" Jul 07 12:51:11.365585 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.365541 2568 scope.go:117] "RemoveContainer" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" Jul 07 12:51:11.365766 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.365749 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9"} err="failed to get container status \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": rpc error: code = NotFound desc = could not find container \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": container with ID starting with 3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9 not found: ID does not exist" Jul 07 12:51:11.365818 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.365766 2568 scope.go:117] "RemoveContainer" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" Jul 07 12:51:11.365980 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.365959 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f"} err="failed to get container status \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": rpc error: code = NotFound desc = could not find container \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": container with ID starting with 1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f not found: ID does not exist" Jul 07 12:51:11.366021 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.365982 2568 scope.go:117] "RemoveContainer" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" Jul 07 12:51:11.366206 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.366187 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c"} err="failed to get container status \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": rpc error: code = NotFound desc = could not find container \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": container with ID starting with bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c not found: ID does not exist" Jul 07 12:51:11.366284 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.366207 2568 scope.go:117] "RemoveContainer" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" Jul 07 12:51:11.366440 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.366419 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb"} err="failed to get container status \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": rpc error: code = NotFound desc = could not find container \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": container with ID starting with d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb not found: ID does not exist" Jul 07 12:51:11.366440 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.366438 2568 scope.go:117] "RemoveContainer" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" Jul 07 12:51:11.366716 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.366698 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7"} err="failed to get container status \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": rpc error: code = NotFound desc = could not find container \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": container with ID starting with 21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7 not found: ID does not exist" Jul 07 12:51:11.366771 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.366717 2568 scope.go:117] "RemoveContainer" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" Jul 07 12:51:11.366947 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.366929 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f"} err="failed to get container status \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": rpc error: code = NotFound desc = could not find container \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": container with ID starting with d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f not found: ID does not exist" Jul 07 12:51:11.367006 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.366949 2568 scope.go:117] "RemoveContainer" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" Jul 07 12:51:11.367197 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367179 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7"} err="failed to get container status \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": rpc error: code = NotFound desc = could not find container \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": container with ID starting with d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7 not found: ID does not exist" Jul 07 12:51:11.367249 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367198 2568 scope.go:117] "RemoveContainer" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" Jul 07 12:51:11.367386 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367365 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9"} err="failed to get container status \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": rpc error: code = NotFound desc = could not find container \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": container with ID starting with 3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9 not found: ID does not exist" Jul 07 12:51:11.367421 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367387 2568 scope.go:117] "RemoveContainer" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" Jul 07 12:51:11.367556 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367536 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f"} err="failed to get container status \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": rpc error: code = NotFound desc = could not find container \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": container with ID starting with 1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f not found: ID does not exist" Jul 07 12:51:11.367602 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367557 2568 scope.go:117] "RemoveContainer" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" Jul 07 12:51:11.367773 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367756 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c"} err="failed to get container status \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": rpc error: code = NotFound desc = could not find container \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": container with ID starting with bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c not found: ID does not exist" Jul 07 12:51:11.367773 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367773 2568 scope.go:117] "RemoveContainer" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" Jul 07 12:51:11.367945 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367930 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb"} err="failed to get container status \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": rpc error: code = NotFound desc = could not find container \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": container with ID starting with d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb not found: ID does not exist" Jul 07 12:51:11.367988 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.367947 2568 scope.go:117] "RemoveContainer" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" Jul 07 12:51:11.368152 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.368101 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7"} err="failed to get container status \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": rpc error: code = NotFound desc = could not find container \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": container with ID starting with 21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7 not found: ID does not exist" Jul 07 12:51:11.368261 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.368194 2568 scope.go:117] "RemoveContainer" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" Jul 07 12:51:11.368431 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.368404 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f"} err="failed to get container status \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": rpc error: code = NotFound desc = could not find container \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": container with ID starting with d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f not found: ID does not exist" Jul 07 12:51:11.368482 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.368432 2568 scope.go:117] "RemoveContainer" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" Jul 07 12:51:11.368676 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.368654 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7"} err="failed to get container status \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": rpc error: code = NotFound desc = could not find container \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": container with ID starting with d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7 not found: ID does not exist" Jul 07 12:51:11.368731 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.368678 2568 scope.go:117] "RemoveContainer" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" Jul 07 12:51:11.368964 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.368943 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9"} err="failed to get container status \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": rpc error: code = NotFound desc = could not find container \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": container with ID starting with 3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9 not found: ID does not exist" Jul 07 12:51:11.368964 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.368963 2568 scope.go:117] "RemoveContainer" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" Jul 07 12:51:11.369206 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.369187 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f"} err="failed to get container status \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": rpc error: code = NotFound desc = could not find container \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": container with ID starting with 1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f not found: ID does not exist" Jul 07 12:51:11.369251 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.369207 2568 scope.go:117] "RemoveContainer" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" Jul 07 12:51:11.369414 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.369398 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c"} err="failed to get container status \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": rpc error: code = NotFound desc = could not find container \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": container with ID starting with bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c not found: ID does not exist" Jul 07 12:51:11.369462 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.369415 2568 scope.go:117] "RemoveContainer" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" Jul 07 12:51:11.369626 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.369602 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb"} err="failed to get container status \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": rpc error: code = NotFound desc = could not find container \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": container with ID starting with d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb not found: ID does not exist" Jul 07 12:51:11.369626 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.369625 2568 scope.go:117] "RemoveContainer" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" Jul 07 12:51:11.369872 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.369855 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7"} err="failed to get container status \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": rpc error: code = NotFound desc = could not find container \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": container with ID starting with 21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7 not found: ID does not exist" Jul 07 12:51:11.369911 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.369872 2568 scope.go:117] "RemoveContainer" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" Jul 07 12:51:11.370054 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370038 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f"} err="failed to get container status \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": rpc error: code = NotFound desc = could not find container \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": container with ID starting with d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f not found: ID does not exist" Jul 07 12:51:11.370097 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370055 2568 scope.go:117] "RemoveContainer" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" Jul 07 12:51:11.370241 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370224 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7"} err="failed to get container status \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": rpc error: code = NotFound desc = could not find container \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": container with ID starting with d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7 not found: ID does not exist" Jul 07 12:51:11.370281 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370240 2568 scope.go:117] "RemoveContainer" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" Jul 07 12:51:11.370456 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370438 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9"} err="failed to get container status \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": rpc error: code = NotFound desc = could not find container \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": container with ID starting with 3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9 not found: ID does not exist" Jul 07 12:51:11.370456 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370454 2568 scope.go:117] "RemoveContainer" containerID="1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f" Jul 07 12:51:11.370666 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370651 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f"} err="failed to get container status \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": rpc error: code = NotFound desc = could not find container \"1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f\": container with ID starting with 1d275d114db9ec382bd698af53c2ccdbccbf9a3593e89931b09aedd7c50cb20f not found: ID does not exist" Jul 07 12:51:11.370666 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370665 2568 scope.go:117] "RemoveContainer" containerID="bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c" Jul 07 12:51:11.370874 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370858 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c"} err="failed to get container status \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": rpc error: code = NotFound desc = could not find container \"bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c\": container with ID starting with bacf63f77842f2e6ebaa914e11051524ac280afc472e6931671eececbd3bfd8c not found: ID does not exist" Jul 07 12:51:11.370913 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.370876 2568 scope.go:117] "RemoveContainer" containerID="d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb" Jul 07 12:51:11.371065 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371046 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb"} err="failed to get container status \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": rpc error: code = NotFound desc = could not find container \"d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb\": container with ID starting with d7115a32f9a1a23b3b29f15be4a428f815743c338ec37f179afd83b4c4373efb not found: ID does not exist" Jul 07 12:51:11.371106 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371066 2568 scope.go:117] "RemoveContainer" containerID="21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7" Jul 07 12:51:11.371300 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371284 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7"} err="failed to get container status \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": rpc error: code = NotFound desc = could not find container \"21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7\": container with ID starting with 21b1c9b6a9a948c293cb3eca745415dd0d97ec61e3cb1740a5789576cd7e6ed7 not found: ID does not exist" Jul 07 12:51:11.371343 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371301 2568 scope.go:117] "RemoveContainer" containerID="d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f" Jul 07 12:51:11.371487 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371471 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f"} err="failed to get container status \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": rpc error: code = NotFound desc = could not find container \"d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f\": container with ID starting with d63da98a7f811b5e5c7427da439ef3bf9c405071341d6e1c92f61bd607bc5c8f not found: ID does not exist" Jul 07 12:51:11.371527 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371489 2568 scope.go:117] "RemoveContainer" containerID="d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7" Jul 07 12:51:11.371709 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371690 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7"} err="failed to get container status \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": rpc error: code = NotFound desc = could not find container \"d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7\": container with ID starting with d89c1c724a852ca51abd6e1c3dee88a37a442b1a1036ac7da078c5fd9acf83a7 not found: ID does not exist" Jul 07 12:51:11.371709 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371708 2568 scope.go:117] "RemoveContainer" containerID="3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9" Jul 07 12:51:11.371910 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.371894 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9"} err="failed to get container status \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": rpc error: code = NotFound desc = could not find container \"3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9\": container with ID starting with 3034e1d777583ada312b7ca88f0db6fab935f3a1e597382d97f3cfee085e16b9 not found: ID does not exist" Jul 07 12:51:11.382848 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.382827 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:51:11.383105 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383091 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="config-reloader" Jul 07 12:51:11.383174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383108 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="config-reloader" Jul 07 12:51:11.383174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383144 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="init-config-reloader" Jul 07 12:51:11.383174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383153 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="init-config-reloader" Jul 07 12:51:11.383174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383163 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="prometheus" Jul 07 12:51:11.383174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383171 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="prometheus" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383181 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy-thanos" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383189 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy-thanos" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383200 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy-web" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383209 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy-web" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383223 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383232 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383245 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="thanos-sidecar" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383251 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="thanos-sidecar" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383301 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="thanos-sidecar" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383309 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy-thanos" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383320 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="config-reloader" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383329 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy-web" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383341 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="prometheus" Jul 07 12:51:11.383353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.383350 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" containerName="kube-rbac-proxy" Jul 07 12:51:11.388896 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.388880 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.395566 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395538 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 12:51:11.395840 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395571 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 12:51:11.395840 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395586 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 12:51:11.395840 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395654 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 12:51:11.395840 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395725 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 12:51:11.396041 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395887 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 12:51:11.396041 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395929 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 12:51:11.396041 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395950 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 12:51:11.396041 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.395931 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 12:51:11.396041 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.396015 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4320ub25glgnn\"" Jul 07 12:51:11.396277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.396051 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 12:51:11.396277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.396200 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 12:51:11.396277 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.396215 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 12:51:11.396430 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.396277 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6cppc\"" Jul 07 12:51:11.396430 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.396220 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 12:51:11.400254 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.400003 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 12:51:11.404173 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.404151 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 12:51:11.411571 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.411551 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:51:11.511812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.511780 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc4a7d88-effd-43b0-9d05-f57f838092fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.511812 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.511814 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z462\" (UniqueName: \"kubernetes.io/projected/dc4a7d88-effd-43b0-9d05-f57f838092fe-kube-api-access-8z462\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.511835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.511916 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.511945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.511966 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512000 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.511984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512040 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512083 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512130 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512185 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512203 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512221 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512221 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512458 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512243 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d88-effd-43b0-9d05-f57f838092fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512458 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512263 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512458 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512458 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.512458 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.512336 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613449 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613524 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613511 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613691 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613691 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613651 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613781 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613781 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613742 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613781 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613765 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613917 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d88-effd-43b0-9d05-f57f838092fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613917 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613917 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613842 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613917 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.613917 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613951 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc4a7d88-effd-43b0-9d05-f57f838092fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.613976 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z462\" (UniqueName: \"kubernetes.io/projected/dc4a7d88-effd-43b0-9d05-f57f838092fe-kube-api-access-8z462\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.614016 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.614052 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.614071 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.614107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614456 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.614184 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614456 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.614212 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.614806 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.614777 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.615094 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.615071 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.615818 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.615793 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.616793 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.616873 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.616940 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.616948 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617288 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.617033 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617578 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.617475 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617578 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.617485 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617578 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.617506 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.617713 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.617684 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.618223 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.618204 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.618505 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.618483 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d88-effd-43b0-9d05-f57f838092fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.619182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.619159 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.619583 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.619567 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d88-effd-43b0-9d05-f57f838092fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.620181 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.620164 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc4a7d88-effd-43b0-9d05-f57f838092fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.620818 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.620799 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc4a7d88-effd-43b0-9d05-f57f838092fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.626560 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.626538 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z462\" (UniqueName: \"kubernetes.io/projected/dc4a7d88-effd-43b0-9d05-f57f838092fe-kube-api-access-8z462\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d88-effd-43b0-9d05-f57f838092fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.698089 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.698054 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:51:11.854270 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:11.854246 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:51:11.855773 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:51:11.855746 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc4a7d88_effd_43b0_9d05_f57f838092fe.slice/crio-b6ff6933e2ef868e2d04d4e624059e6a7e1880d73d1ddfd4f4bc63cec891b3b6 WatchSource:0}: Error finding container b6ff6933e2ef868e2d04d4e624059e6a7e1880d73d1ddfd4f4bc63cec891b3b6: Status 404 returned error can't find the container with id b6ff6933e2ef868e2d04d4e624059e6a7e1880d73d1ddfd4f4bc63cec891b3b6 Jul 07 12:51:12.321392 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:12.321359 2568 generic.go:356] "Generic (PLEG): container finished" podID="dc4a7d88-effd-43b0-9d05-f57f838092fe" containerID="7a5a71a23369147af4b403a529a0a36601e524cf244fa90aa28d05e751428de7" exitCode=0 Jul 07 12:51:12.321773 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:12.321443 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d88-effd-43b0-9d05-f57f838092fe","Type":"ContainerDied","Data":"7a5a71a23369147af4b403a529a0a36601e524cf244fa90aa28d05e751428de7"} Jul 07 12:51:12.321773 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:12.321514 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d88-effd-43b0-9d05-f57f838092fe","Type":"ContainerStarted","Data":"b6ff6933e2ef868e2d04d4e624059e6a7e1880d73d1ddfd4f4bc63cec891b3b6"} Jul 07 12:51:12.603149 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:12.603101 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2fa50a8-958c-454b-911f-d9a50486a67c" path="/var/lib/kubelet/pods/a2fa50a8-958c-454b-911f-d9a50486a67c/volumes" Jul 07 12:51:13.329625 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:13.329593 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d88-effd-43b0-9d05-f57f838092fe","Type":"ContainerStarted","Data":"1b22c44f42b4e5fb6c539b212c201e8051c0f6f4fb34fb242c8018eaf0e02f4f"} Jul 07 12:51:13.329946 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:13.329682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d88-effd-43b0-9d05-f57f838092fe","Type":"ContainerStarted","Data":"9cca14f78b77cadd4d58d3d4e2c2952b05c20063efe2601178515db5ddaae1b0"} Jul 07 12:51:13.329946 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:13.329709 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d88-effd-43b0-9d05-f57f838092fe","Type":"ContainerStarted","Data":"412738fb630fea41b65ef3629f6de92e22b2fdefa9de593b5f931ae050231bd3"} Jul 07 12:51:13.329946 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:13.329739 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d88-effd-43b0-9d05-f57f838092fe","Type":"ContainerStarted","Data":"ec3d42806a8886779f8d3a02a5b0efc84c5d9abe4052e928ca2781b83666cf17"} Jul 07 12:51:13.329946 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:13.329763 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d88-effd-43b0-9d05-f57f838092fe","Type":"ContainerStarted","Data":"72cb4f93ef9e470ae4b8ecb14981020c5fadee60074de47c6b08b30ee4b0b3da"} Jul 07 12:51:13.329946 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:13.329794 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d88-effd-43b0-9d05-f57f838092fe","Type":"ContainerStarted","Data":"265d0978dbfb4f8d70a2fbd9c5dcc506feeb15c1ee25efd6a677f0a9ed6ac72d"} Jul 07 12:51:16.699028 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:51:16.698994 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:52:11.698940 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:52:11.698894 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:52:11.714254 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:52:11.714226 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:52:11.761395 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:52:11.761333 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=60.743790228 podStartE2EDuration="1m0.761310733s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.520505ms" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:13.375337274 +0000 UTC m=+155.360185747" watchObservedRunningTime="2026-07-07 12:52:11.761310733 +0000 UTC m=+213.746159206" Jul 07 12:52:12.555182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:52:12.555152 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:53:38.470577 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:53:38.470549 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 12:54:06.648621 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.648583 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pgjww"] Jul 07 12:54:06.650761 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.650745 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-pgjww" Jul 07 12:54:06.653337 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.653318 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 12:54:06.654298 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.654279 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-l8r4s\"" Jul 07 12:54:06.654412 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.654280 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 12:54:06.662058 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.662037 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pgjww"] Jul 07 12:54:06.814802 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.814765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbqbt\" (UniqueName: \"kubernetes.io/projected/f5ae10cf-e88c-4123-aca2-22281e3f18fd-kube-api-access-xbqbt\") pod \"cert-manager-55b4d954d5-pgjww\" (UID: \"f5ae10cf-e88c-4123-aca2-22281e3f18fd\") " pod="cert-manager/cert-manager-55b4d954d5-pgjww" Jul 07 12:54:06.814971 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.814809 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5ae10cf-e88c-4123-aca2-22281e3f18fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-pgjww\" (UID: \"f5ae10cf-e88c-4123-aca2-22281e3f18fd\") " pod="cert-manager/cert-manager-55b4d954d5-pgjww" Jul 07 12:54:06.915521 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.915439 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbqbt\" (UniqueName: \"kubernetes.io/projected/f5ae10cf-e88c-4123-aca2-22281e3f18fd-kube-api-access-xbqbt\") pod \"cert-manager-55b4d954d5-pgjww\" (UID: \"f5ae10cf-e88c-4123-aca2-22281e3f18fd\") " pod="cert-manager/cert-manager-55b4d954d5-pgjww" Jul 07 12:54:06.915521 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.915481 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5ae10cf-e88c-4123-aca2-22281e3f18fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-pgjww\" (UID: \"f5ae10cf-e88c-4123-aca2-22281e3f18fd\") " pod="cert-manager/cert-manager-55b4d954d5-pgjww" Jul 07 12:54:06.924193 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.924165 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5ae10cf-e88c-4123-aca2-22281e3f18fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-pgjww\" (UID: \"f5ae10cf-e88c-4123-aca2-22281e3f18fd\") " pod="cert-manager/cert-manager-55b4d954d5-pgjww" Jul 07 12:54:06.924361 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.924343 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbqbt\" (UniqueName: \"kubernetes.io/projected/f5ae10cf-e88c-4123-aca2-22281e3f18fd-kube-api-access-xbqbt\") pod \"cert-manager-55b4d954d5-pgjww\" (UID: \"f5ae10cf-e88c-4123-aca2-22281e3f18fd\") " pod="cert-manager/cert-manager-55b4d954d5-pgjww" Jul 07 12:54:06.970671 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:06.970636 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-pgjww" Jul 07 12:54:07.096041 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.096011 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pgjww"] Jul 07 12:54:07.100332 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:54:07.100305 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5ae10cf_e88c_4123_aca2_22281e3f18fd.slice/crio-6baddfe9fe66d63c319c1bd7d5c02add1a6e224f6d5c073334eb8a843f134959 WatchSource:0}: Error finding container 6baddfe9fe66d63c319c1bd7d5c02add1a6e224f6d5c073334eb8a843f134959: Status 404 returned error can't find the container with id 6baddfe9fe66d63c319c1bd7d5c02add1a6e224f6d5c073334eb8a843f134959 Jul 07 12:54:07.102782 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.102765 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:54:07.435039 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.435008 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg"] Jul 07 12:54:07.437672 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.437655 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" Jul 07 12:54:07.442924 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.442898 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 12:54:07.443159 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.443144 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:54:07.444028 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.444004 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-98mll\"" Jul 07 12:54:07.449156 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.449134 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg"] Jul 07 12:54:07.621547 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.621513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px9fw\" (UniqueName: \"kubernetes.io/projected/f7da946a-f6ea-4e65-b953-98e34fb6198c-kube-api-access-px9fw\") pod \"openshift-lws-operator-fd8ccff4c-gjppg\" (UID: \"f7da946a-f6ea-4e65-b953-98e34fb6198c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" Jul 07 12:54:07.621547 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.621547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7da946a-f6ea-4e65-b953-98e34fb6198c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gjppg\" (UID: \"f7da946a-f6ea-4e65-b953-98e34fb6198c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" Jul 07 12:54:07.722917 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.722878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px9fw\" (UniqueName: \"kubernetes.io/projected/f7da946a-f6ea-4e65-b953-98e34fb6198c-kube-api-access-px9fw\") pod \"openshift-lws-operator-fd8ccff4c-gjppg\" (UID: \"f7da946a-f6ea-4e65-b953-98e34fb6198c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" Jul 07 12:54:07.723528 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.722921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7da946a-f6ea-4e65-b953-98e34fb6198c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gjppg\" (UID: \"f7da946a-f6ea-4e65-b953-98e34fb6198c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" Jul 07 12:54:07.723528 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.723394 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7da946a-f6ea-4e65-b953-98e34fb6198c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gjppg\" (UID: \"f7da946a-f6ea-4e65-b953-98e34fb6198c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" Jul 07 12:54:07.731447 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.731423 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px9fw\" (UniqueName: \"kubernetes.io/projected/f7da946a-f6ea-4e65-b953-98e34fb6198c-kube-api-access-px9fw\") pod \"openshift-lws-operator-fd8ccff4c-gjppg\" (UID: \"f7da946a-f6ea-4e65-b953-98e34fb6198c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" Jul 07 12:54:07.747336 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.747317 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" Jul 07 12:54:07.895460 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.895401 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg"] Jul 07 12:54:07.898696 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:54:07.898663 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7da946a_f6ea_4e65_b953_98e34fb6198c.slice/crio-6b675093d4e030ee1bbb556da721c30d431ad5f3feab2abc9331a12f5aadbbda WatchSource:0}: Error finding container 6b675093d4e030ee1bbb556da721c30d431ad5f3feab2abc9331a12f5aadbbda: Status 404 returned error can't find the container with id 6b675093d4e030ee1bbb556da721c30d431ad5f3feab2abc9331a12f5aadbbda Jul 07 12:54:07.952363 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.952315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" event={"ID":"f7da946a-f6ea-4e65-b953-98e34fb6198c","Type":"ContainerStarted","Data":"6b675093d4e030ee1bbb556da721c30d431ad5f3feab2abc9331a12f5aadbbda"} Jul 07 12:54:07.954199 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:07.954154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-pgjww" event={"ID":"f5ae10cf-e88c-4123-aca2-22281e3f18fd","Type":"ContainerStarted","Data":"6baddfe9fe66d63c319c1bd7d5c02add1a6e224f6d5c073334eb8a843f134959"} Jul 07 12:54:10.971686 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:10.971637 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-pgjww" event={"ID":"f5ae10cf-e88c-4123-aca2-22281e3f18fd","Type":"ContainerStarted","Data":"cb9660ba370777c9e2bdca085076c648d514d336890716112eb5dacf19c050b9"} Jul 07 12:54:10.989006 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:10.988955 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-pgjww" podStartSLOduration=1.8595292209999998 podStartE2EDuration="4.988936273s" totalImagesPullingTime="3.129407052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:54:10.988735896 +0000 UTC m=+332.973584368" watchObservedRunningTime="2026-07-07 12:54:10.988936273 +0000 UTC m=+332.973784746" Jul 07 12:54:12.981738 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:12.981702 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" event={"ID":"f7da946a-f6ea-4e65-b953-98e34fb6198c","Type":"ContainerStarted","Data":"9cb9c1a8547790a54d9f3d746a20840003a93f4d949f39011500d5aa9101bf05"} Jul 07 12:54:12.998862 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:12.998816 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gjppg" podStartSLOduration=1.709899228 podStartE2EDuration="5.998802355s" totalImagesPullingTime="4.288903127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:54:12.998467354 +0000 UTC m=+334.983315827" watchObservedRunningTime="2026-07-07 12:54:12.998802355 +0000 UTC m=+334.983650826" Jul 07 12:54:28.614979 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.614944 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq"] Jul 07 12:54:28.621050 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.621035 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.628311 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.628294 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 12:54:28.629636 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.629616 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 12:54:28.629722 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.629616 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 12:54:28.629722 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.629699 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 12:54:28.629950 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.629933 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gknqs\"" Jul 07 12:54:28.653434 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.653410 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq"] Jul 07 12:54:28.685395 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.685370 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-manifests\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.685504 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.685399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.685504 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.685423 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkbq7\" (UniqueName: \"kubernetes.io/projected/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-kube-api-access-zkbq7\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.685504 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.685447 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.685614 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.685536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-tmp\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.785871 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.785846 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-tmp\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.785994 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.785901 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-manifests\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.785994 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.785922 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.785994 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.785955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkbq7\" (UniqueName: \"kubernetes.io/projected/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-kube-api-access-zkbq7\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.785994 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.785991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.786372 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.786351 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-manifests\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.788190 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.788168 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-tmp\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.788367 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.788349 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.788417 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.788370 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.803295 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.803273 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkbq7\" (UniqueName: \"kubernetes.io/projected/33fe06f2-92cc-4940-bf64-0f769f2bf3ac-kube-api-access-zkbq7\") pod \"opendatahub-operator-controller-manager-7c99b98b8b-x6xkq\" (UID: \"33fe06f2-92cc-4940-bf64-0f769f2bf3ac\") " pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:28.930067 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:28.930010 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:29.060742 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:29.060715 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq"] Jul 07 12:54:29.063506 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:54:29.063478 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fe06f2_92cc_4940_bf64_0f769f2bf3ac.slice/crio-4088d9244ee11022774ee459d39f5804c800a8f3f293cb60732907949894d3c8 WatchSource:0}: Error finding container 4088d9244ee11022774ee459d39f5804c800a8f3f293cb60732907949894d3c8: Status 404 returned error can't find the container with id 4088d9244ee11022774ee459d39f5804c800a8f3f293cb60732907949894d3c8 Jul 07 12:54:30.049217 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:30.049180 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" event={"ID":"33fe06f2-92cc-4940-bf64-0f769f2bf3ac","Type":"ContainerStarted","Data":"4088d9244ee11022774ee459d39f5804c800a8f3f293cb60732907949894d3c8"} Jul 07 12:54:33.064350 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:33.064315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" event={"ID":"33fe06f2-92cc-4940-bf64-0f769f2bf3ac","Type":"ContainerStarted","Data":"5cd7a1f2e8565a59f30fe7f0925c9b11dcdad7e1d1c391a1d9e8d8c6cc229a3f"} Jul 07 12:54:34.069750 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:34.069716 2568 generic.go:356] "Generic (PLEG): container finished" podID="33fe06f2-92cc-4940-bf64-0f769f2bf3ac" containerID="5cd7a1f2e8565a59f30fe7f0925c9b11dcdad7e1d1c391a1d9e8d8c6cc229a3f" exitCode=0 Jul 07 12:54:34.070152 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:34.069807 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" event={"ID":"33fe06f2-92cc-4940-bf64-0f769f2bf3ac","Type":"ContainerDied","Data":"5cd7a1f2e8565a59f30fe7f0925c9b11dcdad7e1d1c391a1d9e8d8c6cc229a3f"} Jul 07 12:54:35.074899 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:35.074861 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" event={"ID":"33fe06f2-92cc-4940-bf64-0f769f2bf3ac","Type":"ContainerStarted","Data":"27ac3fe21ae83046d944ee5e1333359662b516fbdab093d9c4e86c19821ba2bc"} Jul 07 12:54:35.075273 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:35.075006 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:35.098565 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:35.098521 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" podStartSLOduration=2.391023614 podStartE2EDuration="7.098507827s" totalImagesPullingTime="3.702375668s" totalInitContainerRuntime="1.005108545s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:54:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:54:35.098090743 +0000 UTC m=+357.082939226" watchObservedRunningTime="2026-07-07 12:54:35.098507827 +0000 UTC m=+357.083356299" Jul 07 12:54:46.081457 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:46.081426 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c99b98b8b-x6xkq" Jul 07 12:54:48.606298 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.606270 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn"] Jul 07 12:54:48.614711 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.614693 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.622796 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.622776 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 12:54:48.622897 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.622871 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 12:54:48.631071 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.631053 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8n8qs\"" Jul 07 12:54:48.631229 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.631213 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 12:54:48.631275 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.631243 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 12:54:48.671189 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.671164 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn"] Jul 07 12:54:48.744663 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.744633 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn8kn\" (UniqueName: \"kubernetes.io/projected/9eae0218-d422-4ae8-b498-5829c3878c36-kube-api-access-gn8kn\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.744780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.744686 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eae0218-d422-4ae8-b498-5829c3878c36-tmp\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.744780 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.744752 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eae0218-d422-4ae8-b498-5829c3878c36-tls-certs\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.845371 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.845343 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eae0218-d422-4ae8-b498-5829c3878c36-tmp\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.845473 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.845385 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eae0218-d422-4ae8-b498-5829c3878c36-tls-certs\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.845473 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.845434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn8kn\" (UniqueName: \"kubernetes.io/projected/9eae0218-d422-4ae8-b498-5829c3878c36-kube-api-access-gn8kn\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.847673 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.847646 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eae0218-d422-4ae8-b498-5829c3878c36-tmp\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.847790 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.847773 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eae0218-d422-4ae8-b498-5829c3878c36-tls-certs\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.870723 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.870674 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn8kn\" (UniqueName: \"kubernetes.io/projected/9eae0218-d422-4ae8-b498-5829c3878c36-kube-api-access-gn8kn\") pod \"kube-auth-proxy-677fdffd75-mzmwn\" (UID: \"9eae0218-d422-4ae8-b498-5829c3878c36\") " pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:48.923213 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:48.923188 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" Jul 07 12:54:49.093447 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:49.093279 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn"] Jul 07 12:54:49.095945 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:54:49.095910 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eae0218_d422_4ae8_b498_5829c3878c36.slice/crio-f7dcda1b3607520c1817656f6a01382467653114b582e9ba588413e31f490293 WatchSource:0}: Error finding container f7dcda1b3607520c1817656f6a01382467653114b582e9ba588413e31f490293: Status 404 returned error can't find the container with id f7dcda1b3607520c1817656f6a01382467653114b582e9ba588413e31f490293 Jul 07 12:54:49.128824 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:49.128761 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" event={"ID":"9eae0218-d422-4ae8-b498-5829c3878c36","Type":"ContainerStarted","Data":"f7dcda1b3607520c1817656f6a01382467653114b582e9ba588413e31f490293"} Jul 07 12:54:53.146594 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:53.146553 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" event={"ID":"9eae0218-d422-4ae8-b498-5829c3878c36","Type":"ContainerStarted","Data":"1898a29d338a53eeff6ff5460f97f85bf5088c46be37b316dda4a411eb15c00d"} Jul 07 12:54:53.165977 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:53.165929 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-677fdffd75-mzmwn" podStartSLOduration=1.98510069 podStartE2EDuration="5.165914426s" totalImagesPullingTime="3.180813736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:54:53.164366257 +0000 UTC m=+375.149214739" watchObservedRunningTime="2026-07-07 12:54:53.165914426 +0000 UTC m=+375.150763124" Jul 07 12:54:55.871762 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:55.871729 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx"] Jul 07 12:54:55.874892 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:55.874876 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:55.877845 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:55.877827 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 12:54:55.877970 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:55.877830 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 12:54:55.878182 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:55.878165 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 12:54:55.878243 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:55.878165 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 12:54:55.878243 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:55.878172 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-twjc8\"" Jul 07 12:54:55.892853 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:55.892822 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx"] Jul 07 12:54:56.003648 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.003621 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.003801 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.003665 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.003801 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.003688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mpz4\" (UniqueName: \"kubernetes.io/projected/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-kube-api-access-6mpz4\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.003801 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.003723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.003978 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.003799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.003978 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.003833 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.003978 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.003870 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.104718 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.104688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.104853 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.104730 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.104853 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.104752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mpz4\" (UniqueName: \"kubernetes.io/projected/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-kube-api-access-6mpz4\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.104853 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.104773 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.104853 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.104803 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.104853 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.104824 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.104853 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.104850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.105562 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.105535 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.107098 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.107080 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.107200 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.107105 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.107200 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.107162 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.107272 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.107249 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.124768 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.124717 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mpz4\" (UniqueName: \"kubernetes.io/projected/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-kube-api-access-6mpz4\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.132871 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.132854 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c2c61d0-8953-4228-9446-dcb4b2bb73cd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-57qwx\" (UID: \"8c2c61d0-8953-4228-9446-dcb4b2bb73cd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.184032 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.184011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:56.319559 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:56.317877 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx"] Jul 07 12:54:56.322789 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:54:56.322761 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2c61d0_8953_4228_9446_dcb4b2bb73cd.slice/crio-f4fd66fe7ed53227487fbe0201e8e9e39ef951e0b444951c6a46f8bd0d818abf WatchSource:0}: Error finding container f4fd66fe7ed53227487fbe0201e8e9e39ef951e0b444951c6a46f8bd0d818abf: Status 404 returned error can't find the container with id f4fd66fe7ed53227487fbe0201e8e9e39ef951e0b444951c6a46f8bd0d818abf Jul 07 12:54:57.167713 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:57.167677 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" event={"ID":"8c2c61d0-8953-4228-9446-dcb4b2bb73cd","Type":"ContainerStarted","Data":"f4fd66fe7ed53227487fbe0201e8e9e39ef951e0b444951c6a46f8bd0d818abf"} Jul 07 12:54:58.698633 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:58.698593 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 07 12:54:58.698997 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:58.698662 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 07 12:54:59.178559 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.178524 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" event={"ID":"8c2c61d0-8953-4228-9446-dcb4b2bb73cd","Type":"ContainerStarted","Data":"4a9ff43212cdb816c78319e1081ce9b6b6de30fb71578727666427eee62999f8"} Jul 07 12:54:59.178732 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.178682 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:54:59.180353 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.180330 2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-57qwx container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 12:54:59.180465 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.180381 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" podUID="8c2c61d0-8953-4228-9446-dcb4b2bb73cd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 12:54:59.770790 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.770729 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" podStartSLOduration=2.397062426 podStartE2EDuration="4.770708064s" totalImagesPullingTime="2.373645638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:54:59.31924727 +0000 UTC m=+381.304095741" watchObservedRunningTime="2026-07-07 12:54:59.770708064 +0000 UTC m=+381.755556537" Jul 07 12:54:59.772675 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.772640 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99"] Jul 07 12:54:59.776174 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.776157 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.782463 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.782433 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 12:54:59.782569 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.782434 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 12:54:59.782569 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.782438 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 12:54:59.782569 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.782517 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tfpcr\"" Jul 07 12:54:59.819371 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.819343 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99"] Jul 07 12:54:59.836957 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.836930 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-manager-config\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.837073 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.836965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-metrics-cert\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.837073 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.837038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-cert\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.837073 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.837061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82dwf\" (UniqueName: \"kubernetes.io/projected/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-kube-api-access-82dwf\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.937553 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.937517 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-manager-config\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.937713 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.937571 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-metrics-cert\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.937713 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.937635 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-cert\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.937713 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.937664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82dwf\" (UniqueName: \"kubernetes.io/projected/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-kube-api-access-82dwf\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.938310 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.938288 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-manager-config\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.940203 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.940178 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-cert\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.940339 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.940324 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-metrics-cert\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:54:59.976245 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:54:59.976214 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82dwf\" (UniqueName: \"kubernetes.io/projected/bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb-kube-api-access-82dwf\") pod \"lws-controller-manager-7c57499d8c-62c99\" (UID: \"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb\") " pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:55:00.085620 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:55:00.085545 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:55:00.185498 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:55:00.185431 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-57qwx" Jul 07 12:55:00.252231 ip-10-0-131-17 kubenswrapper[2568]: W0707 12:55:00.252206 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4209d9_4c7b_4d88_9adc_cd9cc4c50bfb.slice/crio-cb336f91c89baf875f52e628ea26e4f52a1c5aac76e5bdc596a7a4d48ae36435 WatchSource:0}: Error finding container cb336f91c89baf875f52e628ea26e4f52a1c5aac76e5bdc596a7a4d48ae36435: Status 404 returned error can't find the container with id cb336f91c89baf875f52e628ea26e4f52a1c5aac76e5bdc596a7a4d48ae36435 Jul 07 12:55:00.260035 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:55:00.260011 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99"] Jul 07 12:55:01.191411 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:55:01.191372 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" event={"ID":"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb","Type":"ContainerStarted","Data":"cb336f91c89baf875f52e628ea26e4f52a1c5aac76e5bdc596a7a4d48ae36435"} Jul 07 12:55:03.200438 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:55:03.200398 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" event={"ID":"bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb","Type":"ContainerStarted","Data":"a226d6b99ae5da04f9fec9d2058a18383f73d78d80334f8fe5cdb3a1a7987ef0"} Jul 07 12:55:03.200775 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:55:03.200520 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 12:55:03.227526 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:55:03.227483 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" podStartSLOduration=2.180720248 podStartE2EDuration="4.227469141s" totalImagesPullingTime="2.046748893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:54:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:55:03.226215745 +0000 UTC m=+385.211064219" watchObservedRunningTime="2026-07-07 12:55:03.227469141 +0000 UTC m=+385.212317614" Jul 07 12:55:14.207070 ip-10-0-131-17 kubenswrapper[2568]: I0707 12:55:14.207036 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c57499d8c-62c99" Jul 07 13:14:51.938110 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:51.938075 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq"] Jul 07 13:14:51.941309 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:51.941288 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:51.944190 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:51.944169 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-bafec94a-serving-cert\"" Jul 07 13:14:51.944292 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:51.944204 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:14:51.953753 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:51.953734 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq"] Jul 07 13:14:52.044764 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.044741 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz59b\" (UniqueName: \"kubernetes.io/projected/99dadf71-3d81-48b7-a08a-38f7422d65f1-kube-api-access-tz59b\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.044873 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.044786 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.044917 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.044875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dadf71-3d81-48b7-a08a-38f7422d65f1-openshift-service-ca\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.145707 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.145682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dadf71-3d81-48b7-a08a-38f7422d65f1-openshift-service-ca\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.145820 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.145732 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz59b\" (UniqueName: \"kubernetes.io/projected/99dadf71-3d81-48b7-a08a-38f7422d65f1-kube-api-access-tz59b\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.145820 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.145769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.145894 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:14:52.145877 2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-bafec94a-serving-cert: secret "maas-api-e2e-ait-bafec94a-serving-cert" not found Jul 07 13:14:52.145951 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:14:52.145942 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls podName:99dadf71-3d81-48b7-a08a-38f7422d65f1 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:52.645925245 +0000 UTC m=+1574.630773699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls") pod "maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" (UID: "99dadf71-3d81-48b7-a08a-38f7422d65f1") : secret "maas-api-e2e-ait-bafec94a-serving-cert" not found Jul 07 13:14:52.146394 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.146374 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dadf71-3d81-48b7-a08a-38f7422d65f1-openshift-service-ca\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.161614 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.161594 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz59b\" (UniqueName: \"kubernetes.io/projected/99dadf71-3d81-48b7-a08a-38f7422d65f1-kube-api-access-tz59b\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.649981 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.649953 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.652253 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.652231 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls\") pod \"maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.852020 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.851991 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:52.976140 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.976089 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq"] Jul 07 13:14:52.978974 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:14:52.978950 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99dadf71_3d81_48b7_a08a_38f7422d65f1.slice/crio-ee271ecfc9cc772c3252841c188890114640545bd7cb4e029b67cf43606eace9 WatchSource:0}: Error finding container ee271ecfc9cc772c3252841c188890114640545bd7cb4e029b67cf43606eace9: Status 404 returned error can't find the container with id ee271ecfc9cc772c3252841c188890114640545bd7cb4e029b67cf43606eace9 Jul 07 13:14:52.980701 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:52.980686 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:14:53.187820 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:53.187741 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" event={"ID":"99dadf71-3d81-48b7-a08a-38f7422d65f1","Type":"ContainerStarted","Data":"ee271ecfc9cc772c3252841c188890114640545bd7cb4e029b67cf43606eace9"} Jul 07 13:14:56.202751 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:56.202711 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" event={"ID":"99dadf71-3d81-48b7-a08a-38f7422d65f1","Type":"ContainerStarted","Data":"a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4"} Jul 07 13:14:56.203236 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:56.202813 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:56.221645 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:56.221597 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" podStartSLOduration=3.055395827 podStartE2EDuration="5.221583496s" totalImagesPullingTime="2.166187669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:56.220751431 +0000 UTC m=+1578.205599906" watchObservedRunningTime="2026-07-07 13:14:56.221583496 +0000 UTC m=+1578.206431967" Jul 07 13:14:57.246830 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:57.246793 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq"] Jul 07 13:14:57.292730 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:14:57.292688 2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-bafec94a-serving-cert: secret "maas-api-e2e-ait-bafec94a-serving-cert" not found Jul 07 13:14:57.292893 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:14:57.292767 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls podName:99dadf71-3d81-48b7-a08a-38f7422d65f1 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:57.792750549 +0000 UTC m=+1579.777598999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls") pod "maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" (UID: "99dadf71-3d81-48b7-a08a-38f7422d65f1") : secret "maas-api-e2e-ait-bafec94a-serving-cert" not found Jul 07 13:14:57.795851 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:14:57.795818 2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-bafec94a-serving-cert: secret "maas-api-e2e-ait-bafec94a-serving-cert" not found Jul 07 13:14:57.796014 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:14:57.795882 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls podName:99dadf71-3d81-48b7-a08a-38f7422d65f1 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:58.795866972 +0000 UTC m=+1580.780715422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls") pod "maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" (UID: "99dadf71-3d81-48b7-a08a-38f7422d65f1") : secret "maas-api-e2e-ait-bafec94a-serving-cert" not found Jul 07 13:14:58.212879 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.212770 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" podUID="99dadf71-3d81-48b7-a08a-38f7422d65f1" containerName="maas-api" containerID="cri-o://a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4" gracePeriod=30 Jul 07 13:14:58.447739 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.447716 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:58.500098 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.500074 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dadf71-3d81-48b7-a08a-38f7422d65f1-openshift-service-ca\") pod \"99dadf71-3d81-48b7-a08a-38f7422d65f1\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " Jul 07 13:14:58.500290 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.500160 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz59b\" (UniqueName: \"kubernetes.io/projected/99dadf71-3d81-48b7-a08a-38f7422d65f1-kube-api-access-tz59b\") pod \"99dadf71-3d81-48b7-a08a-38f7422d65f1\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " Jul 07 13:14:58.500290 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.500206 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls\") pod \"99dadf71-3d81-48b7-a08a-38f7422d65f1\" (UID: \"99dadf71-3d81-48b7-a08a-38f7422d65f1\") " Jul 07 13:14:58.500427 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.500406 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99dadf71-3d81-48b7-a08a-38f7422d65f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99dadf71-3d81-48b7-a08a-38f7422d65f1" (UID: "99dadf71-3d81-48b7-a08a-38f7422d65f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:14:58.502341 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.502314 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99dadf71-3d81-48b7-a08a-38f7422d65f1" (UID: "99dadf71-3d81-48b7-a08a-38f7422d65f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:14:58.502443 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.502350 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99dadf71-3d81-48b7-a08a-38f7422d65f1-kube-api-access-tz59b" (OuterVolumeSpecName: "kube-api-access-tz59b") pod "99dadf71-3d81-48b7-a08a-38f7422d65f1" (UID: "99dadf71-3d81-48b7-a08a-38f7422d65f1"). InnerVolumeSpecName "kube-api-access-tz59b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:14:58.600691 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.600663 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dadf71-3d81-48b7-a08a-38f7422d65f1-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:14:58.600691 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.600686 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz59b\" (UniqueName: \"kubernetes.io/projected/99dadf71-3d81-48b7-a08a-38f7422d65f1-kube-api-access-tz59b\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:14:58.600691 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:58.600696 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dadf71-3d81-48b7-a08a-38f7422d65f1-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:14:59.218609 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.218575 2568 generic.go:356] "Generic (PLEG): container finished" podID="99dadf71-3d81-48b7-a08a-38f7422d65f1" containerID="a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4" exitCode=0 Jul 07 13:14:59.218775 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.218619 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" Jul 07 13:14:59.218775 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.218617 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" event={"ID":"99dadf71-3d81-48b7-a08a-38f7422d65f1","Type":"ContainerDied","Data":"a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4"} Jul 07 13:14:59.218775 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.218748 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq" event={"ID":"99dadf71-3d81-48b7-a08a-38f7422d65f1","Type":"ContainerDied","Data":"ee271ecfc9cc772c3252841c188890114640545bd7cb4e029b67cf43606eace9"} Jul 07 13:14:59.218874 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.218779 2568 scope.go:117] "RemoveContainer" containerID="a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4" Jul 07 13:14:59.227054 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.227002 2568 scope.go:117] "RemoveContainer" containerID="a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4" Jul 07 13:14:59.227276 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:14:59.227247 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4\": container with ID starting with a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4 not found: ID does not exist" containerID="a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4" Jul 07 13:14:59.227332 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.227283 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4"} err="failed to get container status \"a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4\": rpc error: code = NotFound desc = could not find container \"a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4\": container with ID starting with a082409f2f0492235667a6ce1b9e0ada711a57b86fd8073545974c736c43b7d4 not found: ID does not exist" Jul 07 13:14:59.235340 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.235319 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq"] Jul 07 13:14:59.237559 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:14:59.237539 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-bafec94a-7464ffdbc9-qggjq"] Jul 07 13:15:00.128391 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.128356 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723835-7nsbr"] Jul 07 13:15:00.128767 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.128698 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99dadf71-3d81-48b7-a08a-38f7422d65f1" containerName="maas-api" Jul 07 13:15:00.128767 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.128709 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="99dadf71-3d81-48b7-a08a-38f7422d65f1" containerName="maas-api" Jul 07 13:15:00.128835 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.128772 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="99dadf71-3d81-48b7-a08a-38f7422d65f1" containerName="maas-api" Jul 07 13:15:00.133182 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.133156 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" Jul 07 13:15:00.135921 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.135897 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:15:00.138430 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.138396 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723835-7nsbr"] Jul 07 13:15:00.212211 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.212177 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjm6x\" (UniqueName: \"kubernetes.io/projected/b3794214-6812-4402-8aee-359bbf2d5dad-kube-api-access-wjm6x\") pod \"maas-api-key-cleanup-29723835-7nsbr\" (UID: \"b3794214-6812-4402-8aee-359bbf2d5dad\") " pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" Jul 07 13:15:00.312876 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.312834 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjm6x\" (UniqueName: \"kubernetes.io/projected/b3794214-6812-4402-8aee-359bbf2d5dad-kube-api-access-wjm6x\") pod \"maas-api-key-cleanup-29723835-7nsbr\" (UID: \"b3794214-6812-4402-8aee-359bbf2d5dad\") " pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" Jul 07 13:15:00.321154 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.321100 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjm6x\" (UniqueName: \"kubernetes.io/projected/b3794214-6812-4402-8aee-359bbf2d5dad-kube-api-access-wjm6x\") pod \"maas-api-key-cleanup-29723835-7nsbr\" (UID: \"b3794214-6812-4402-8aee-359bbf2d5dad\") " pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" Jul 07 13:15:00.444985 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.444897 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" Jul 07 13:15:00.602985 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.602953 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99dadf71-3d81-48b7-a08a-38f7422d65f1" path="/var/lib/kubelet/pods/99dadf71-3d81-48b7-a08a-38f7422d65f1/volumes" Jul 07 13:15:00.780298 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:00.780270 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723835-7nsbr"] Jul 07 13:15:00.783312 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:15:00.783283 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3794214_6812_4402_8aee_359bbf2d5dad.slice/crio-4372e4280bfa8c6a4ce32dec90d6a7c960e117f7f5e103ffd109dcfe1ea9fa47 WatchSource:0}: Error finding container 4372e4280bfa8c6a4ce32dec90d6a7c960e117f7f5e103ffd109dcfe1ea9fa47: Status 404 returned error can't find the container with id 4372e4280bfa8c6a4ce32dec90d6a7c960e117f7f5e103ffd109dcfe1ea9fa47 Jul 07 13:15:01.229472 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:01.229437 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" event={"ID":"b3794214-6812-4402-8aee-359bbf2d5dad","Type":"ContainerStarted","Data":"4372e4280bfa8c6a4ce32dec90d6a7c960e117f7f5e103ffd109dcfe1ea9fa47"} Jul 07 13:15:03.239927 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:03.239895 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" event={"ID":"b3794214-6812-4402-8aee-359bbf2d5dad","Type":"ContainerStarted","Data":"e3637a4a029fd9853643a5d3f1bcc809d70c3a6f9c1c0571bda14ae25549a785"} Jul 07 13:15:03.258995 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:03.258941 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" podStartSLOduration=1.296026482 podStartE2EDuration="3.258924428s" totalImagesPullingTime="1.962897946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:03.258573926 +0000 UTC m=+1585.243422411" watchObservedRunningTime="2026-07-07 13:15:03.258924428 +0000 UTC m=+1585.243772900" Jul 07 13:15:04.245547 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:04.245506 2568 generic.go:356] "Generic (PLEG): container finished" podID="b3794214-6812-4402-8aee-359bbf2d5dad" containerID="e3637a4a029fd9853643a5d3f1bcc809d70c3a6f9c1c0571bda14ae25549a785" exitCode=0 Jul 07 13:15:04.245902 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:04.245594 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" event={"ID":"b3794214-6812-4402-8aee-359bbf2d5dad","Type":"ContainerDied","Data":"e3637a4a029fd9853643a5d3f1bcc809d70c3a6f9c1c0571bda14ae25549a785"} Jul 07 13:15:05.002703 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.002672 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk"] Jul 07 13:15:05.005952 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.005929 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.008890 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.008872 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-b0651b4d-serving-cert\"" Jul 07 13:15:05.022390 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.022366 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk"] Jul 07 13:15:05.051040 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.051019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64lcd\" (UniqueName: \"kubernetes.io/projected/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-kube-api-access-64lcd\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.051152 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.051055 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-openshift-service-ca\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.051152 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.051075 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.152295 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.152272 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64lcd\" (UniqueName: \"kubernetes.io/projected/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-kube-api-access-64lcd\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.152388 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.152312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-openshift-service-ca\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.152388 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.152333 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.152502 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:15:05.152487 2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-b0651b4d-serving-cert: secret "maas-api-e2e-ait-b0651b4d-serving-cert" not found Jul 07 13:15:05.152554 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:15:05.152545 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls podName:04509d92-f3e1-4e0e-b5b3-dd1189fbed2e nodeName:}" failed. No retries permitted until 2026-07-07 13:15:05.652528644 +0000 UTC m=+1587.637377095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls") pod "maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" (UID: "04509d92-f3e1-4e0e-b5b3-dd1189fbed2e") : secret "maas-api-e2e-ait-b0651b4d-serving-cert" not found Jul 07 13:15:05.152923 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.152905 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-openshift-service-ca\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.164043 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.164016 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64lcd\" (UniqueName: \"kubernetes.io/projected/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-kube-api-access-64lcd\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.377389 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.377368 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" Jul 07 13:15:05.455611 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.455579 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjm6x\" (UniqueName: \"kubernetes.io/projected/b3794214-6812-4402-8aee-359bbf2d5dad-kube-api-access-wjm6x\") pod \"b3794214-6812-4402-8aee-359bbf2d5dad\" (UID: \"b3794214-6812-4402-8aee-359bbf2d5dad\") " Jul 07 13:15:05.457482 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.457460 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3794214-6812-4402-8aee-359bbf2d5dad-kube-api-access-wjm6x" (OuterVolumeSpecName: "kube-api-access-wjm6x") pod "b3794214-6812-4402-8aee-359bbf2d5dad" (UID: "b3794214-6812-4402-8aee-359bbf2d5dad"). InnerVolumeSpecName "kube-api-access-wjm6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:05.556409 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.556351 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjm6x\" (UniqueName: \"kubernetes.io/projected/b3794214-6812-4402-8aee-359bbf2d5dad-kube-api-access-wjm6x\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:15:05.657142 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.657093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.659481 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.659460 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls\") pod \"maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:05.915617 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:05.915542 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:06.043755 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:06.043729 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk"] Jul 07 13:15:06.045611 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:15:06.045583 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04509d92_f3e1_4e0e_b5b3_dd1189fbed2e.slice/crio-fa1e4e9fe6f7ecf5cf0d7811b645ecfd28022e753868463b36e9927011a1250c WatchSource:0}: Error finding container fa1e4e9fe6f7ecf5cf0d7811b645ecfd28022e753868463b36e9927011a1250c: Status 404 returned error can't find the container with id fa1e4e9fe6f7ecf5cf0d7811b645ecfd28022e753868463b36e9927011a1250c Jul 07 13:15:06.255521 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:06.255490 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" event={"ID":"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e","Type":"ContainerStarted","Data":"fa1e4e9fe6f7ecf5cf0d7811b645ecfd28022e753868463b36e9927011a1250c"} Jul 07 13:15:06.256581 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:06.256559 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" event={"ID":"b3794214-6812-4402-8aee-359bbf2d5dad","Type":"ContainerDied","Data":"4372e4280bfa8c6a4ce32dec90d6a7c960e117f7f5e103ffd109dcfe1ea9fa47"} Jul 07 13:15:06.256696 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:06.256589 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723835-7nsbr" Jul 07 13:15:06.256696 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:06.256598 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4372e4280bfa8c6a4ce32dec90d6a7c960e117f7f5e103ffd109dcfe1ea9fa47" Jul 07 13:15:07.262668 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:07.262636 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" event={"ID":"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e","Type":"ContainerStarted","Data":"94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe"} Jul 07 13:15:07.263019 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:07.262759 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:07.294519 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:07.294466 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" podStartSLOduration=2.760928129 podStartE2EDuration="3.294453072s" totalImagesPullingTime="533.524943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:07.293052899 +0000 UTC m=+1589.277901372" watchObservedRunningTime="2026-07-07 13:15:07.294453072 +0000 UTC m=+1589.279301545" Jul 07 13:15:10.255817 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.255784 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk"] Jul 07 13:15:10.256333 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.255959 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" podUID="04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" containerName="maas-api" containerID="cri-o://94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe" gracePeriod=30 Jul 07 13:15:10.496895 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.496872 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:10.596026 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.595953 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls\") pod \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " Jul 07 13:15:10.596026 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.596005 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64lcd\" (UniqueName: \"kubernetes.io/projected/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-kube-api-access-64lcd\") pod \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " Jul 07 13:15:10.596213 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.596033 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-openshift-service-ca\") pod \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\" (UID: \"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e\") " Jul 07 13:15:10.596464 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.596440 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" (UID: "04509d92-f3e1-4e0e-b5b3-dd1189fbed2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:10.598062 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.598037 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" (UID: "04509d92-f3e1-4e0e-b5b3-dd1189fbed2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:15:10.598177 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.598140 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-kube-api-access-64lcd" (OuterVolumeSpecName: "kube-api-access-64lcd") pod "04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" (UID: "04509d92-f3e1-4e0e-b5b3-dd1189fbed2e"). InnerVolumeSpecName "kube-api-access-64lcd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:10.697268 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.697238 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:15:10.697268 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.697262 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64lcd\" (UniqueName: \"kubernetes.io/projected/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-kube-api-access-64lcd\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:15:10.697268 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:10.697272 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:15:11.281746 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.281705 2568 generic.go:356] "Generic (PLEG): container finished" podID="04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" containerID="94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe" exitCode=0 Jul 07 13:15:11.281746 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.281749 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" Jul 07 13:15:11.282227 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.281794 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" event={"ID":"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e","Type":"ContainerDied","Data":"94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe"} Jul 07 13:15:11.282227 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.281845 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk" event={"ID":"04509d92-f3e1-4e0e-b5b3-dd1189fbed2e","Type":"ContainerDied","Data":"fa1e4e9fe6f7ecf5cf0d7811b645ecfd28022e753868463b36e9927011a1250c"} Jul 07 13:15:11.282227 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.281869 2568 scope.go:117] "RemoveContainer" containerID="94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe" Jul 07 13:15:11.290561 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.290538 2568 scope.go:117] "RemoveContainer" containerID="94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe" Jul 07 13:15:11.290859 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:15:11.290834 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe\": container with ID starting with 94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe not found: ID does not exist" containerID="94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe" Jul 07 13:15:11.290983 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.290867 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe"} err="failed to get container status \"94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe\": rpc error: code = NotFound desc = could not find container \"94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe\": container with ID starting with 94a054e7f3d2a06fe5fe3cf34b904582debfa68ad92ef30f7c82d28d056393fe not found: ID does not exist" Jul 07 13:15:11.300498 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.300473 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk"] Jul 07 13:15:11.304786 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:11.304766 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-b0651b4d-dbd8566d6-xmdsk"] Jul 07 13:15:12.603802 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:12.603764 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" path="/var/lib/kubelet/pods/04509d92-f3e1-4e0e-b5b3-dd1189fbed2e/volumes" Jul 07 13:15:23.642498 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.642462 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg"] Jul 07 13:15:23.642969 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.642952 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" containerName="maas-api" Jul 07 13:15:23.643018 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.642972 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" containerName="maas-api" Jul 07 13:15:23.643018 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.642983 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3794214-6812-4402-8aee-359bbf2d5dad" containerName="cleanup" Jul 07 13:15:23.643018 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.642988 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3794214-6812-4402-8aee-359bbf2d5dad" containerName="cleanup" Jul 07 13:15:23.643130 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.643055 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3794214-6812-4402-8aee-359bbf2d5dad" containerName="cleanup" Jul 07 13:15:23.643130 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.643064 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="04509d92-f3e1-4e0e-b5b3-dd1189fbed2e" containerName="maas-api" Jul 07 13:15:23.648622 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.648606 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:23.651205 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.651185 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:15:23.651878 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.651864 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-34ce5f66-serving-cert\"" Jul 07 13:15:23.659311 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.659286 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg"] Jul 07 13:15:23.704487 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.704465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:23.704587 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.704517 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f848453d-621d-4476-9b26-1db1c058ea0f-openshift-service-ca\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:23.704587 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.704573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82v87\" (UniqueName: \"kubernetes.io/projected/f848453d-621d-4476-9b26-1db1c058ea0f-kube-api-access-82v87\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:23.805760 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.805729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:23.805864 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.805789 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f848453d-621d-4476-9b26-1db1c058ea0f-openshift-service-ca\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:23.805864 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.805838 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82v87\" (UniqueName: \"kubernetes.io/projected/f848453d-621d-4476-9b26-1db1c058ea0f-kube-api-access-82v87\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:23.805945 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:15:23.805907 2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-34ce5f66-serving-cert: secret "maas-api-e2e-derive-34ce5f66-serving-cert" not found Jul 07 13:15:23.805996 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:15:23.805985 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls podName:f848453d-621d-4476-9b26-1db1c058ea0f nodeName:}" failed. No retries permitted until 2026-07-07 13:15:24.305968635 +0000 UTC m=+1606.290817085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls") pod "maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" (UID: "f848453d-621d-4476-9b26-1db1c058ea0f") : secret "maas-api-e2e-derive-34ce5f66-serving-cert" not found Jul 07 13:15:23.806482 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.806466 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f848453d-621d-4476-9b26-1db1c058ea0f-openshift-service-ca\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:23.827874 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:23.827854 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82v87\" (UniqueName: \"kubernetes.io/projected/f848453d-621d-4476-9b26-1db1c058ea0f-kube-api-access-82v87\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:24.310402 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:24.310371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:24.312744 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:24.312712 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls\") pod \"maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:24.559233 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:24.559199 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:24.895397 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:24.895325 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg"] Jul 07 13:15:24.898673 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:15:24.898646 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf848453d_621d_4476_9b26_1db1c058ea0f.slice/crio-d238c090fb9457e49c2ffd2f268128f9c57b6bc12c449a51aa9a5d0e952ad04c WatchSource:0}: Error finding container d238c090fb9457e49c2ffd2f268128f9c57b6bc12c449a51aa9a5d0e952ad04c: Status 404 returned error can't find the container with id d238c090fb9457e49c2ffd2f268128f9c57b6bc12c449a51aa9a5d0e952ad04c Jul 07 13:15:25.344439 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:25.344413 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" event={"ID":"f848453d-621d-4476-9b26-1db1c058ea0f","Type":"ContainerStarted","Data":"be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d"} Jul 07 13:15:25.344539 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:25.344459 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" event={"ID":"f848453d-621d-4476-9b26-1db1c058ea0f","Type":"ContainerStarted","Data":"d238c090fb9457e49c2ffd2f268128f9c57b6bc12c449a51aa9a5d0e952ad04c"} Jul 07 13:15:25.344599 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:25.344536 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:25.365686 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:25.365648 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" podStartSLOduration=2.005557577 podStartE2EDuration="2.365635215s" totalImagesPullingTime="360.077638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:25.36418113 +0000 UTC m=+1607.349029603" watchObservedRunningTime="2026-07-07 13:15:25.365635215 +0000 UTC m=+1607.350483686" Jul 07 13:15:28.340468 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.340423 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg"] Jul 07 13:15:28.340987 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.340668 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" podUID="f848453d-621d-4476-9b26-1db1c058ea0f" containerName="maas-api" containerID="cri-o://be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d" gracePeriod=30 Jul 07 13:15:28.599171 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.599087 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:28.744988 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.744945 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82v87\" (UniqueName: \"kubernetes.io/projected/f848453d-621d-4476-9b26-1db1c058ea0f-kube-api-access-82v87\") pod \"f848453d-621d-4476-9b26-1db1c058ea0f\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " Jul 07 13:15:28.745215 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.745004 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f848453d-621d-4476-9b26-1db1c058ea0f-openshift-service-ca\") pod \"f848453d-621d-4476-9b26-1db1c058ea0f\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " Jul 07 13:15:28.745215 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.745045 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls\") pod \"f848453d-621d-4476-9b26-1db1c058ea0f\" (UID: \"f848453d-621d-4476-9b26-1db1c058ea0f\") " Jul 07 13:15:28.745467 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.745442 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f848453d-621d-4476-9b26-1db1c058ea0f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f848453d-621d-4476-9b26-1db1c058ea0f" (UID: "f848453d-621d-4476-9b26-1db1c058ea0f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:28.747036 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.747017 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f848453d-621d-4476-9b26-1db1c058ea0f" (UID: "f848453d-621d-4476-9b26-1db1c058ea0f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:15:28.747101 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.747076 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f848453d-621d-4476-9b26-1db1c058ea0f-kube-api-access-82v87" (OuterVolumeSpecName: "kube-api-access-82v87") pod "f848453d-621d-4476-9b26-1db1c058ea0f" (UID: "f848453d-621d-4476-9b26-1db1c058ea0f"). InnerVolumeSpecName "kube-api-access-82v87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:28.846029 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.846006 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f848453d-621d-4476-9b26-1db1c058ea0f-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:15:28.846029 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.846028 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82v87\" (UniqueName: \"kubernetes.io/projected/f848453d-621d-4476-9b26-1db1c058ea0f-kube-api-access-82v87\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:15:28.846183 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:28.846037 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f848453d-621d-4476-9b26-1db1c058ea0f-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:15:29.361599 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.361556 2568 generic.go:356] "Generic (PLEG): container finished" podID="f848453d-621d-4476-9b26-1db1c058ea0f" containerID="be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d" exitCode=0 Jul 07 13:15:29.361599 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.361602 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" Jul 07 13:15:29.362056 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.361647 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" event={"ID":"f848453d-621d-4476-9b26-1db1c058ea0f","Type":"ContainerDied","Data":"be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d"} Jul 07 13:15:29.362056 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.361704 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg" event={"ID":"f848453d-621d-4476-9b26-1db1c058ea0f","Type":"ContainerDied","Data":"d238c090fb9457e49c2ffd2f268128f9c57b6bc12c449a51aa9a5d0e952ad04c"} Jul 07 13:15:29.362056 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.361725 2568 scope.go:117] "RemoveContainer" containerID="be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d" Jul 07 13:15:29.370217 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.370202 2568 scope.go:117] "RemoveContainer" containerID="be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d" Jul 07 13:15:29.370465 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:15:29.370446 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d\": container with ID starting with be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d not found: ID does not exist" containerID="be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d" Jul 07 13:15:29.370531 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.370473 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d"} err="failed to get container status \"be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d\": rpc error: code = NotFound desc = could not find container \"be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d\": container with ID starting with be0cd5a1bc5c86264c7b219eba84ddf885eb484e477aa12de367db531f6a8a4d not found: ID does not exist" Jul 07 13:15:29.383532 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.383511 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg"] Jul 07 13:15:29.388293 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:29.388269 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-34ce5f66-b97cf595c-nwgzg"] Jul 07 13:15:30.602759 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:15:30.602730 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f848453d-621d-4476-9b26-1db1c058ea0f" path="/var/lib/kubelet/pods/f848453d-621d-4476-9b26-1db1c058ea0f/volumes" Jul 07 13:17:14.991640 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:14.991561 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj"] Jul 07 13:17:14.992079 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:14.991883 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f848453d-621d-4476-9b26-1db1c058ea0f" containerName="maas-api" Jul 07 13:17:14.992079 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:14.991893 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f848453d-621d-4476-9b26-1db1c058ea0f" containerName="maas-api" Jul 07 13:17:14.992079 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:14.991952 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f848453d-621d-4476-9b26-1db1c058ea0f" containerName="maas-api" Jul 07 13:17:14.994898 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:14.994882 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:14.997808 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:14.997787 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-d75b40cf-serving-cert\"" Jul 07 13:17:14.998391 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:14.998366 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:17:15.007499 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.007479 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj"] Jul 07 13:17:15.101750 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.101715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-openshift-service-ca\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.101890 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.101773 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbrnv\" (UniqueName: \"kubernetes.io/projected/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-kube-api-access-wbrnv\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.101890 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.101878 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.202721 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.202690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-openshift-service-ca\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.202862 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.202739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrnv\" (UniqueName: \"kubernetes.io/projected/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-kube-api-access-wbrnv\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.202862 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.202789 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.202936 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:17:15.202893 2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-d75b40cf-serving-cert: secret "maas-api-e2e-mt-d75b40cf-serving-cert" not found Jul 07 13:17:15.202978 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:17:15.202968 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls podName:8dbcdcfe-0f58-4f41-b528-5a035070a9d0 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:15.702946926 +0000 UTC m=+1717.687795376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls") pod "maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" (UID: "8dbcdcfe-0f58-4f41-b528-5a035070a9d0") : secret "maas-api-e2e-mt-d75b40cf-serving-cert" not found Jul 07 13:17:15.203431 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.203412 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-openshift-service-ca\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.220217 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.220187 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbrnv\" (UniqueName: \"kubernetes.io/projected/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-kube-api-access-wbrnv\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.705673 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.705638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.708013 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.707983 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls\") pod \"maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:15.904887 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:15.904858 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:16.033965 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:16.033910 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj"] Jul 07 13:17:16.036254 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:17:16.036226 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbcdcfe_0f58_4f41_b528_5a035070a9d0.slice/crio-d8375ef2213c3528a84b44b15986e01b41e5866498b93bf2f3489d624227dc68 WatchSource:0}: Error finding container d8375ef2213c3528a84b44b15986e01b41e5866498b93bf2f3489d624227dc68: Status 404 returned error can't find the container with id d8375ef2213c3528a84b44b15986e01b41e5866498b93bf2f3489d624227dc68 Jul 07 13:17:16.818716 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:16.818683 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" event={"ID":"8dbcdcfe-0f58-4f41-b528-5a035070a9d0","Type":"ContainerStarted","Data":"74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824"} Jul 07 13:17:16.818861 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:16.818734 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" event={"ID":"8dbcdcfe-0f58-4f41-b528-5a035070a9d0","Type":"ContainerStarted","Data":"d8375ef2213c3528a84b44b15986e01b41e5866498b93bf2f3489d624227dc68"} Jul 07 13:17:16.818861 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:16.818820 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:16.836916 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:16.836869 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" podStartSLOduration=2.419432301 podStartE2EDuration="2.836856236s" totalImagesPullingTime="417.423935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:16.835777263 +0000 UTC m=+1718.820625735" watchObservedRunningTime="2026-07-07 13:17:16.836856236 +0000 UTC m=+1718.821704708" Jul 07 13:17:22.830092 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:22.830062 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:27.228900 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.228852 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj"] Jul 07 13:17:27.229386 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.229096 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" podUID="8dbcdcfe-0f58-4f41-b528-5a035070a9d0" containerName="maas-api" containerID="cri-o://74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824" gracePeriod=30 Jul 07 13:17:27.480902 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.480844 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:27.610430 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.610393 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-openshift-service-ca\") pod \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " Jul 07 13:17:27.610599 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.610467 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls\") pod \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " Jul 07 13:17:27.610599 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.610514 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbrnv\" (UniqueName: \"kubernetes.io/projected/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-kube-api-access-wbrnv\") pod \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\" (UID: \"8dbcdcfe-0f58-4f41-b528-5a035070a9d0\") " Jul 07 13:17:27.610784 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.610760 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8dbcdcfe-0f58-4f41-b528-5a035070a9d0" (UID: "8dbcdcfe-0f58-4f41-b528-5a035070a9d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:27.612555 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.612530 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8dbcdcfe-0f58-4f41-b528-5a035070a9d0" (UID: "8dbcdcfe-0f58-4f41-b528-5a035070a9d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:17:27.612555 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.612546 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-kube-api-access-wbrnv" (OuterVolumeSpecName: "kube-api-access-wbrnv") pod "8dbcdcfe-0f58-4f41-b528-5a035070a9d0" (UID: "8dbcdcfe-0f58-4f41-b528-5a035070a9d0"). InnerVolumeSpecName "kube-api-access-wbrnv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:17:27.711818 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.711776 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:17:27.711818 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.711812 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:17:27.711818 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.711822 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbrnv\" (UniqueName: \"kubernetes.io/projected/8dbcdcfe-0f58-4f41-b528-5a035070a9d0-kube-api-access-wbrnv\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:17:27.868797 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.868706 2568 generic.go:356] "Generic (PLEG): container finished" podID="8dbcdcfe-0f58-4f41-b528-5a035070a9d0" containerID="74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824" exitCode=0 Jul 07 13:17:27.868797 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.868751 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" Jul 07 13:17:27.868967 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.868791 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" event={"ID":"8dbcdcfe-0f58-4f41-b528-5a035070a9d0","Type":"ContainerDied","Data":"74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824"} Jul 07 13:17:27.868967 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.868842 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj" event={"ID":"8dbcdcfe-0f58-4f41-b528-5a035070a9d0","Type":"ContainerDied","Data":"d8375ef2213c3528a84b44b15986e01b41e5866498b93bf2f3489d624227dc68"} Jul 07 13:17:27.868967 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.868865 2568 scope.go:117] "RemoveContainer" containerID="74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824" Jul 07 13:17:27.877963 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.877946 2568 scope.go:117] "RemoveContainer" containerID="74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824" Jul 07 13:17:27.878261 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:17:27.878243 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824\": container with ID starting with 74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824 not found: ID does not exist" containerID="74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824" Jul 07 13:17:27.878323 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.878270 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824"} err="failed to get container status \"74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824\": rpc error: code = NotFound desc = could not find container \"74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824\": container with ID starting with 74ede5967e373e2bbdcf91b72b86d4370aa0d847a303e683f01182120393f824 not found: ID does not exist" Jul 07 13:17:27.891620 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.891594 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj"] Jul 07 13:17:27.896618 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:27.896595 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-d75b40cf-68989cb698-lp2hj"] Jul 07 13:17:28.603007 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:28.602979 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dbcdcfe-0f58-4f41-b528-5a035070a9d0" path="/var/lib/kubelet/pods/8dbcdcfe-0f58-4f41-b528-5a035070a9d0/volumes" Jul 07 13:17:56.836931 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.836896 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57"] Jul 07 13:17:56.837423 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.837222 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dbcdcfe-0f58-4f41-b528-5a035070a9d0" containerName="maas-api" Jul 07 13:17:56.837423 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.837233 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dbcdcfe-0f58-4f41-b528-5a035070a9d0" containerName="maas-api" Jul 07 13:17:56.837423 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.837294 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dbcdcfe-0f58-4f41-b528-5a035070a9d0" containerName="maas-api" Jul 07 13:17:56.840134 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.840100 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:56.843979 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.843960 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:17:56.844246 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.844230 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-9638b7d0-serving-cert\"" Jul 07 13:17:56.852007 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.851984 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57"] Jul 07 13:17:56.942310 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.942276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad3930-4cea-47da-b613-a4900aac1688-openshift-service-ca\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:56.942501 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.942350 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r78nh\" (UniqueName: \"kubernetes.io/projected/54ad3930-4cea-47da-b613-a4900aac1688-kube-api-access-r78nh\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:56.942501 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:56.942388 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54ad3930-4cea-47da-b613-a4900aac1688-maas-api-tls\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:57.043010 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.042974 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r78nh\" (UniqueName: \"kubernetes.io/projected/54ad3930-4cea-47da-b613-a4900aac1688-kube-api-access-r78nh\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:57.043010 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.043009 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54ad3930-4cea-47da-b613-a4900aac1688-maas-api-tls\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:57.043255 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.043091 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad3930-4cea-47da-b613-a4900aac1688-openshift-service-ca\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:57.043774 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.043753 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad3930-4cea-47da-b613-a4900aac1688-openshift-service-ca\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:57.045597 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.045574 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54ad3930-4cea-47da-b613-a4900aac1688-maas-api-tls\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:57.051736 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.051712 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r78nh\" (UniqueName: \"kubernetes.io/projected/54ad3930-4cea-47da-b613-a4900aac1688-kube-api-access-r78nh\") pod \"maas-api-e2e-mt-9638b7d0-9c778b654-mcs57\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:57.150950 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.150870 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:57.282658 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.282517 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57"] Jul 07 13:17:57.285226 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:17:57.285197 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54ad3930_4cea_47da_b613_a4900aac1688.slice/crio-a6f1d927b1be29536ffe5e232edab0be23b61740d59b877b8da30b6eb3f95c76 WatchSource:0}: Error finding container a6f1d927b1be29536ffe5e232edab0be23b61740d59b877b8da30b6eb3f95c76: Status 404 returned error can't find the container with id a6f1d927b1be29536ffe5e232edab0be23b61740d59b877b8da30b6eb3f95c76 Jul 07 13:17:57.997928 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.997888 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" event={"ID":"54ad3930-4cea-47da-b613-a4900aac1688","Type":"ContainerStarted","Data":"9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4"} Jul 07 13:17:57.998330 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.997948 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" event={"ID":"54ad3930-4cea-47da-b613-a4900aac1688","Type":"ContainerStarted","Data":"a6f1d927b1be29536ffe5e232edab0be23b61740d59b877b8da30b6eb3f95c76"} Jul 07 13:17:57.998330 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:57.998022 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:17:58.016491 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:17:58.016412 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" podStartSLOduration=1.550522562 podStartE2EDuration="2.016398962s" totalImagesPullingTime="465.8764ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:58.015322058 +0000 UTC m=+1760.000170530" watchObservedRunningTime="2026-07-07 13:17:58.016398962 +0000 UTC m=+1760.001247433" Jul 07 13:18:04.006963 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:04.006926 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:18:34.609023 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.608948 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57"] Jul 07 13:18:34.609536 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.609167 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" podUID="54ad3930-4cea-47da-b613-a4900aac1688" containerName="maas-api" containerID="cri-o://9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4" gracePeriod=30 Jul 07 13:18:34.852579 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.852553 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:18:34.950173 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.950096 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad3930-4cea-47da-b613-a4900aac1688-openshift-service-ca\") pod \"54ad3930-4cea-47da-b613-a4900aac1688\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " Jul 07 13:18:34.950283 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.950192 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r78nh\" (UniqueName: \"kubernetes.io/projected/54ad3930-4cea-47da-b613-a4900aac1688-kube-api-access-r78nh\") pod \"54ad3930-4cea-47da-b613-a4900aac1688\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " Jul 07 13:18:34.950283 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.950247 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54ad3930-4cea-47da-b613-a4900aac1688-maas-api-tls\") pod \"54ad3930-4cea-47da-b613-a4900aac1688\" (UID: \"54ad3930-4cea-47da-b613-a4900aac1688\") " Jul 07 13:18:34.950505 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.950484 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54ad3930-4cea-47da-b613-a4900aac1688-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "54ad3930-4cea-47da-b613-a4900aac1688" (UID: "54ad3930-4cea-47da-b613-a4900aac1688"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:18:34.950570 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.950551 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad3930-4cea-47da-b613-a4900aac1688-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:18:34.952236 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.952214 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54ad3930-4cea-47da-b613-a4900aac1688-kube-api-access-r78nh" (OuterVolumeSpecName: "kube-api-access-r78nh") pod "54ad3930-4cea-47da-b613-a4900aac1688" (UID: "54ad3930-4cea-47da-b613-a4900aac1688"). InnerVolumeSpecName "kube-api-access-r78nh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:18:34.952324 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:34.952242 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54ad3930-4cea-47da-b613-a4900aac1688-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "54ad3930-4cea-47da-b613-a4900aac1688" (UID: "54ad3930-4cea-47da-b613-a4900aac1688"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:18:35.051257 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.051226 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r78nh\" (UniqueName: \"kubernetes.io/projected/54ad3930-4cea-47da-b613-a4900aac1688-kube-api-access-r78nh\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:18:35.051257 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.051253 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54ad3930-4cea-47da-b613-a4900aac1688-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:18:35.156567 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.156531 2568 generic.go:356] "Generic (PLEG): container finished" podID="54ad3930-4cea-47da-b613-a4900aac1688" containerID="9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4" exitCode=0 Jul 07 13:18:35.156664 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.156580 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" Jul 07 13:18:35.156664 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.156621 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" event={"ID":"54ad3930-4cea-47da-b613-a4900aac1688","Type":"ContainerDied","Data":"9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4"} Jul 07 13:18:35.156756 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.156677 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57" event={"ID":"54ad3930-4cea-47da-b613-a4900aac1688","Type":"ContainerDied","Data":"a6f1d927b1be29536ffe5e232edab0be23b61740d59b877b8da30b6eb3f95c76"} Jul 07 13:18:35.156756 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.156706 2568 scope.go:117] "RemoveContainer" containerID="9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4" Jul 07 13:18:35.166881 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.166854 2568 scope.go:117] "RemoveContainer" containerID="9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4" Jul 07 13:18:35.167936 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:18:35.167909 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4\": container with ID starting with 9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4 not found: ID does not exist" containerID="9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4" Jul 07 13:18:35.168163 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.168082 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4"} err="failed to get container status \"9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4\": rpc error: code = NotFound desc = could not find container \"9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4\": container with ID starting with 9efff1a3785f8e983497c2609071cca552209433f2ff307f42c89fd41d4a2db4 not found: ID does not exist" Jul 07 13:18:35.190618 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.190594 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57"] Jul 07 13:18:35.195491 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:35.195471 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-9638b7d0-9c778b654-mcs57"] Jul 07 13:18:36.603638 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:18:36.603604 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54ad3930-4cea-47da-b613-a4900aac1688" path="/var/lib/kubelet/pods/54ad3930-4cea-47da-b613-a4900aac1688/volumes" Jul 07 13:20:32.724485 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.724454 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl"] Jul 07 13:20:32.724881 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.724803 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54ad3930-4cea-47da-b613-a4900aac1688" containerName="maas-api" Jul 07 13:20:32.724881 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.724816 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="54ad3930-4cea-47da-b613-a4900aac1688" containerName="maas-api" Jul 07 13:20:32.724881 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.724868 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="54ad3930-4cea-47da-b613-a4900aac1688" containerName="maas-api" Jul 07 13:20:32.727813 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.727796 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:32.731767 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.731748 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:20:32.731983 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.731968 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-5ba2ec-serving-cert\"" Jul 07 13:20:32.756903 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.756878 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl"] Jul 07 13:20:32.898931 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.898900 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l67mq\" (UniqueName: \"kubernetes.io/projected/e0913e55-7add-4165-bd91-77180e74f108-kube-api-access-l67mq\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:32.898931 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.898936 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:32.899141 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.898962 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0913e55-7add-4165-bd91-77180e74f108-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:32.999751 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.999654 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0913e55-7add-4165-bd91-77180e74f108-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:32.999926 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.999774 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l67mq\" (UniqueName: \"kubernetes.io/projected/e0913e55-7add-4165-bd91-77180e74f108-kube-api-access-l67mq\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:32.999926 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:32.999803 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:32.999926 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:20:32.999921 2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-5ba2ec-serving-cert: secret "maas-api-e2e-inf-a-5ba2ec-serving-cert" not found Jul 07 13:20:33.000093 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:20:33.000007 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls podName:e0913e55-7add-4165-bd91-77180e74f108 nodeName:}" failed. No retries permitted until 2026-07-07 13:20:33.499984645 +0000 UTC m=+1915.484833113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls") pod "maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" (UID: "e0913e55-7add-4165-bd91-77180e74f108") : secret "maas-api-e2e-inf-a-5ba2ec-serving-cert" not found Jul 07 13:20:33.000362 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:33.000344 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0913e55-7add-4165-bd91-77180e74f108-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:33.009700 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:33.009679 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l67mq\" (UniqueName: \"kubernetes.io/projected/e0913e55-7add-4165-bd91-77180e74f108-kube-api-access-l67mq\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:33.504753 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:33.504716 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:33.507230 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:33.507204 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls\") pod \"maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:33.638024 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:33.637990 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:33.779865 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:33.779770 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl"] Jul 07 13:20:33.782547 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:20:33.782518 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0913e55_7add_4165_bd91_77180e74f108.slice/crio-320f4562521b2e66954e280592ed448f3142c7082436b5e82d02361183438183 WatchSource:0}: Error finding container 320f4562521b2e66954e280592ed448f3142c7082436b5e82d02361183438183: Status 404 returned error can't find the container with id 320f4562521b2e66954e280592ed448f3142c7082436b5e82d02361183438183 Jul 07 13:20:33.783804 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:33.783783 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:20:34.667847 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:34.667752 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" event={"ID":"e0913e55-7add-4165-bd91-77180e74f108","Type":"ContainerStarted","Data":"beb281914ec5691b6d49047d0df3d9f196d22f7b338842cb679007d3cb61fce0"} Jul 07 13:20:34.667847 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:34.667831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" event={"ID":"e0913e55-7add-4165-bd91-77180e74f108","Type":"ContainerStarted","Data":"320f4562521b2e66954e280592ed448f3142c7082436b5e82d02361183438183"} Jul 07 13:20:34.668066 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:34.667855 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:34.691630 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:34.691581 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" podStartSLOduration=2.259484414 podStartE2EDuration="2.691568313s" totalImagesPullingTime="432.083899ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:20:34.688608319 +0000 UTC m=+1916.673456791" watchObservedRunningTime="2026-07-07 13:20:34.691568313 +0000 UTC m=+1916.676416784" Jul 07 13:20:40.677090 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.677063 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:20:40.771618 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.771580 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj"] Jul 07 13:20:40.774911 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.774889 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.780206 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.780176 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-256cc6-serving-cert\"" Jul 07 13:20:40.802431 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.802408 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj"] Jul 07 13:20:40.869945 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.869915 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33db245-c0dd-4d53-9811-67cd47f0573b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.869945 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.869947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33db245-c0dd-4d53-9811-67cd47f0573b-maas-api-tls\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.870140 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.870028 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvs5q\" (UniqueName: \"kubernetes.io/projected/d33db245-c0dd-4d53-9811-67cd47f0573b-kube-api-access-pvs5q\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.971364 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.971337 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33db245-c0dd-4d53-9811-67cd47f0573b-maas-api-tls\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.971478 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.971385 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvs5q\" (UniqueName: \"kubernetes.io/projected/d33db245-c0dd-4d53-9811-67cd47f0573b-kube-api-access-pvs5q\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.971478 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.971449 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33db245-c0dd-4d53-9811-67cd47f0573b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.972103 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.972077 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33db245-c0dd-4d53-9811-67cd47f0573b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.973826 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.973795 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33db245-c0dd-4d53-9811-67cd47f0573b-maas-api-tls\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:40.981557 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:40.981532 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvs5q\" (UniqueName: \"kubernetes.io/projected/d33db245-c0dd-4d53-9811-67cd47f0573b-kube-api-access-pvs5q\") pod \"maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:41.085438 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:41.085405 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:41.221062 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:41.221041 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj"] Jul 07 13:20:41.223265 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:20:41.223194 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33db245_c0dd_4d53_9811_67cd47f0573b.slice/crio-ac339904d25d7b7fae8e8ed1fd77bca29bbca83d58b336e26656fe90a0237a3d WatchSource:0}: Error finding container ac339904d25d7b7fae8e8ed1fd77bca29bbca83d58b336e26656fe90a0237a3d: Status 404 returned error can't find the container with id ac339904d25d7b7fae8e8ed1fd77bca29bbca83d58b336e26656fe90a0237a3d Jul 07 13:20:41.697057 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:41.697033 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" event={"ID":"d33db245-c0dd-4d53-9811-67cd47f0573b","Type":"ContainerStarted","Data":"ac339904d25d7b7fae8e8ed1fd77bca29bbca83d58b336e26656fe90a0237a3d"} Jul 07 13:20:42.702294 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:42.702259 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" event={"ID":"d33db245-c0dd-4d53-9811-67cd47f0573b","Type":"ContainerStarted","Data":"b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b"} Jul 07 13:20:42.702736 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:42.702409 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:48.712037 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:48.712011 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:20:48.731492 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:20:48.731429 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" podStartSLOduration=8.305904161 podStartE2EDuration="8.731410335s" totalImagesPullingTime="425.506174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:20:42.72203626 +0000 UTC m=+1924.706884732" watchObservedRunningTime="2026-07-07 13:20:48.731410335 +0000 UTC m=+1930.716258808" Jul 07 13:21:45.625141 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:45.625042 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl"] Jul 07 13:21:45.625640 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:45.625317 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" podUID="e0913e55-7add-4165-bd91-77180e74f108" containerName="maas-api" containerID="cri-o://beb281914ec5691b6d49047d0df3d9f196d22f7b338842cb679007d3cb61fce0" gracePeriod=30 Jul 07 13:21:45.672823 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:45.672793 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" podUID="e0913e55-7add-4165-bd91-77180e74f108" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.29:8443/health\": dial tcp 10.134.0.29:8443: connect: connection refused" Jul 07 13:21:45.974888 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:45.974862 2568 generic.go:356] "Generic (PLEG): container finished" podID="e0913e55-7add-4165-bd91-77180e74f108" containerID="beb281914ec5691b6d49047d0df3d9f196d22f7b338842cb679007d3cb61fce0" exitCode=0 Jul 07 13:21:45.975012 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:45.974934 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" event={"ID":"e0913e55-7add-4165-bd91-77180e74f108","Type":"ContainerDied","Data":"beb281914ec5691b6d49047d0df3d9f196d22f7b338842cb679007d3cb61fce0"} Jul 07 13:21:46.069331 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.069306 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:21:46.105044 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.105022 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0913e55-7add-4165-bd91-77180e74f108-openshift-service-ca\") pod \"e0913e55-7add-4165-bd91-77180e74f108\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " Jul 07 13:21:46.105198 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.105056 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l67mq\" (UniqueName: \"kubernetes.io/projected/e0913e55-7add-4165-bd91-77180e74f108-kube-api-access-l67mq\") pod \"e0913e55-7add-4165-bd91-77180e74f108\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " Jul 07 13:21:46.105198 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.105141 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls\") pod \"e0913e55-7add-4165-bd91-77180e74f108\" (UID: \"e0913e55-7add-4165-bd91-77180e74f108\") " Jul 07 13:21:46.105425 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.105402 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0913e55-7add-4165-bd91-77180e74f108-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e0913e55-7add-4165-bd91-77180e74f108" (UID: "e0913e55-7add-4165-bd91-77180e74f108"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:21:46.107095 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.107069 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e0913e55-7add-4165-bd91-77180e74f108" (UID: "e0913e55-7add-4165-bd91-77180e74f108"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:21:46.107200 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.107092 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0913e55-7add-4165-bd91-77180e74f108-kube-api-access-l67mq" (OuterVolumeSpecName: "kube-api-access-l67mq") pod "e0913e55-7add-4165-bd91-77180e74f108" (UID: "e0913e55-7add-4165-bd91-77180e74f108"). InnerVolumeSpecName "kube-api-access-l67mq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:21:46.206295 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.206238 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0913e55-7add-4165-bd91-77180e74f108-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:21:46.206295 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.206260 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l67mq\" (UniqueName: \"kubernetes.io/projected/e0913e55-7add-4165-bd91-77180e74f108-kube-api-access-l67mq\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:21:46.206295 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.206272 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0913e55-7add-4165-bd91-77180e74f108-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:21:46.980190 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.980154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" event={"ID":"e0913e55-7add-4165-bd91-77180e74f108","Type":"ContainerDied","Data":"320f4562521b2e66954e280592ed448f3142c7082436b5e82d02361183438183"} Jul 07 13:21:46.980588 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.980219 2568 scope.go:117] "RemoveContainer" containerID="beb281914ec5691b6d49047d0df3d9f196d22f7b338842cb679007d3cb61fce0" Jul 07 13:21:46.980588 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.980163 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl" Jul 07 13:21:46.999246 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:46.999220 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl"] Jul 07 13:21:47.000592 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:47.000569 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-5ba2ec-6565f6967c-7qtjl"] Jul 07 13:21:48.603723 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:21:48.603690 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0913e55-7add-4165-bd91-77180e74f108" path="/var/lib/kubelet/pods/e0913e55-7add-4165-bd91-77180e74f108/volumes" Jul 07 13:22:28.867183 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:28.867151 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj"] Jul 07 13:22:28.867634 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:28.867350 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" podUID="d33db245-c0dd-4d53-9811-67cd47f0573b" containerName="maas-api" containerID="cri-o://b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b" gracePeriod=30 Jul 07 13:22:29.109325 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.109302 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:22:29.160880 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.160813 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvs5q\" (UniqueName: \"kubernetes.io/projected/d33db245-c0dd-4d53-9811-67cd47f0573b-kube-api-access-pvs5q\") pod \"d33db245-c0dd-4d53-9811-67cd47f0573b\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " Jul 07 13:22:29.160880 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.160865 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33db245-c0dd-4d53-9811-67cd47f0573b-openshift-service-ca\") pod \"d33db245-c0dd-4d53-9811-67cd47f0573b\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " Jul 07 13:22:29.160880 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.160884 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33db245-c0dd-4d53-9811-67cd47f0573b-maas-api-tls\") pod \"d33db245-c0dd-4d53-9811-67cd47f0573b\" (UID: \"d33db245-c0dd-4d53-9811-67cd47f0573b\") " Jul 07 13:22:29.161241 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.161216 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d33db245-c0dd-4d53-9811-67cd47f0573b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d33db245-c0dd-4d53-9811-67cd47f0573b" (UID: "d33db245-c0dd-4d53-9811-67cd47f0573b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:29.163090 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.163062 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d33db245-c0dd-4d53-9811-67cd47f0573b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d33db245-c0dd-4d53-9811-67cd47f0573b" (UID: "d33db245-c0dd-4d53-9811-67cd47f0573b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:22:29.163241 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.163216 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33db245-c0dd-4d53-9811-67cd47f0573b-kube-api-access-pvs5q" (OuterVolumeSpecName: "kube-api-access-pvs5q") pod "d33db245-c0dd-4d53-9811-67cd47f0573b" (UID: "d33db245-c0dd-4d53-9811-67cd47f0573b"). InnerVolumeSpecName "kube-api-access-pvs5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:29.163241 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.163215 2568 generic.go:356] "Generic (PLEG): container finished" podID="d33db245-c0dd-4d53-9811-67cd47f0573b" containerID="b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b" exitCode=0 Jul 07 13:22:29.163351 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.163256 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" Jul 07 13:22:29.163351 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.163291 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" event={"ID":"d33db245-c0dd-4d53-9811-67cd47f0573b","Type":"ContainerDied","Data":"b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b"} Jul 07 13:22:29.163351 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.163342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj" event={"ID":"d33db245-c0dd-4d53-9811-67cd47f0573b","Type":"ContainerDied","Data":"ac339904d25d7b7fae8e8ed1fd77bca29bbca83d58b336e26656fe90a0237a3d"} Jul 07 13:22:29.163471 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.163367 2568 scope.go:117] "RemoveContainer" containerID="b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b" Jul 07 13:22:29.178584 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.178564 2568 scope.go:117] "RemoveContainer" containerID="b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b" Jul 07 13:22:29.178856 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:22:29.178835 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b\": container with ID starting with b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b not found: ID does not exist" containerID="b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b" Jul 07 13:22:29.178920 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.178868 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b"} err="failed to get container status \"b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b\": rpc error: code = NotFound desc = could not find container \"b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b\": container with ID starting with b50783375dc938ba19ef616c1a535d4c02f77e25bc19c271dcceb5ed44d6520b not found: ID does not exist" Jul 07 13:22:29.192164 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.192141 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj"] Jul 07 13:22:29.198954 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.198934 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-256cc6-849cbd5f65-crjrj"] Jul 07 13:22:29.261756 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.261735 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvs5q\" (UniqueName: \"kubernetes.io/projected/d33db245-c0dd-4d53-9811-67cd47f0573b-kube-api-access-pvs5q\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:22:29.261756 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.261757 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d33db245-c0dd-4d53-9811-67cd47f0573b-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:22:29.261898 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:29.261770 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d33db245-c0dd-4d53-9811-67cd47f0573b-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:22:30.602726 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:30.602690 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d33db245-c0dd-4d53-9811-67cd47f0573b" path="/var/lib/kubelet/pods/d33db245-c0dd-4d53-9811-67cd47f0573b/volumes" Jul 07 13:22:38.396099 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.396062 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4"] Jul 07 13:22:38.396581 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.396426 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d33db245-c0dd-4d53-9811-67cd47f0573b" containerName="maas-api" Jul 07 13:22:38.396581 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.396438 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d33db245-c0dd-4d53-9811-67cd47f0573b" containerName="maas-api" Jul 07 13:22:38.396581 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.396463 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0913e55-7add-4165-bd91-77180e74f108" containerName="maas-api" Jul 07 13:22:38.396581 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.396469 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0913e55-7add-4165-bd91-77180e74f108" containerName="maas-api" Jul 07 13:22:38.396581 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.396518 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d33db245-c0dd-4d53-9811-67cd47f0573b" containerName="maas-api" Jul 07 13:22:38.396581 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.396530 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0913e55-7add-4165-bd91-77180e74f108" containerName="maas-api" Jul 07 13:22:38.401106 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.401084 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.404337 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.404312 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-aa07aa-serving-cert\"" Jul 07 13:22:38.404452 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.404421 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:22:38.422538 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.422516 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4"] Jul 07 13:22:38.534837 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.534808 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af294b0d-aec7-4870-92d5-d0790c0fb026-maas-api-tls\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.534979 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.534860 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af294b0d-aec7-4870-92d5-d0790c0fb026-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.534979 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.534931 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtm4v\" (UniqueName: \"kubernetes.io/projected/af294b0d-aec7-4870-92d5-d0790c0fb026-kube-api-access-jtm4v\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.636333 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.636306 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af294b0d-aec7-4870-92d5-d0790c0fb026-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.636492 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.636353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtm4v\" (UniqueName: \"kubernetes.io/projected/af294b0d-aec7-4870-92d5-d0790c0fb026-kube-api-access-jtm4v\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.636492 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.636393 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af294b0d-aec7-4870-92d5-d0790c0fb026-maas-api-tls\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.636987 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.636964 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af294b0d-aec7-4870-92d5-d0790c0fb026-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.639562 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.639543 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-aa07aa-serving-cert\"" Jul 07 13:22:38.645786 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.645763 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtm4v\" (UniqueName: \"kubernetes.io/projected/af294b0d-aec7-4870-92d5-d0790c0fb026-kube-api-access-jtm4v\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.648433 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.648387 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af294b0d-aec7-4870-92d5-d0790c0fb026-maas-api-tls\") pod \"maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.714185 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.714162 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:22:38.721633 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.721615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:38.850853 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:38.850825 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4"] Jul 07 13:22:38.852841 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:22:38.852813 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf294b0d_aec7_4870_92d5_d0790c0fb026.slice/crio-cd0321a66216a1f24999d4b3b5d95286070120896ce961eb09462c3016acdbde WatchSource:0}: Error finding container cd0321a66216a1f24999d4b3b5d95286070120896ce961eb09462c3016acdbde: Status 404 returned error can't find the container with id cd0321a66216a1f24999d4b3b5d95286070120896ce961eb09462c3016acdbde Jul 07 13:22:39.211358 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:39.211312 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" event={"ID":"af294b0d-aec7-4870-92d5-d0790c0fb026","Type":"ContainerStarted","Data":"cd0321a66216a1f24999d4b3b5d95286070120896ce961eb09462c3016acdbde"} Jul 07 13:22:40.216680 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:40.216637 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" event={"ID":"af294b0d-aec7-4870-92d5-d0790c0fb026","Type":"ContainerStarted","Data":"d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090"} Jul 07 13:22:40.217066 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:40.216769 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:40.242036 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:40.241987 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" podStartSLOduration=1.879122625 podStartE2EDuration="2.241968701s" totalImagesPullingTime="362.846076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:40.240588655 +0000 UTC m=+2042.225437146" watchObservedRunningTime="2026-07-07 13:22:40.241968701 +0000 UTC m=+2042.226817174" Jul 07 13:22:46.226168 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.226140 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:46.302281 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.302248 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf"] Jul 07 13:22:46.304809 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.304790 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:46.310163 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.310142 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-619289-serving-cert\"" Jul 07 13:22:46.319340 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.319316 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf"] Jul 07 13:22:46.405788 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.405751 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:46.405915 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.405834 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b567aa-8ca1-4603-9e68-632b19ccceb6-openshift-service-ca\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:46.405915 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.405859 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tlx6\" (UniqueName: \"kubernetes.io/projected/e3b567aa-8ca1-4603-9e68-632b19ccceb6-kube-api-access-9tlx6\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:46.507265 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.507196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:46.507389 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.507276 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b567aa-8ca1-4603-9e68-632b19ccceb6-openshift-service-ca\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:46.507389 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:22:46.507341 2568 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-619289-serving-cert: secret "maas-api-e2e-api-b-619289-serving-cert" not found Jul 07 13:22:46.507469 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:22:46.507410 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls podName:e3b567aa-8ca1-4603-9e68-632b19ccceb6 nodeName:}" failed. No retries permitted until 2026-07-07 13:22:47.007392622 +0000 UTC m=+2048.992241076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls") pod "maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" (UID: "e3b567aa-8ca1-4603-9e68-632b19ccceb6") : secret "maas-api-e2e-api-b-619289-serving-cert" not found Jul 07 13:22:46.507469 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.507433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tlx6\" (UniqueName: \"kubernetes.io/projected/e3b567aa-8ca1-4603-9e68-632b19ccceb6-kube-api-access-9tlx6\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:46.507913 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.507896 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b567aa-8ca1-4603-9e68-632b19ccceb6-openshift-service-ca\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:46.515989 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:46.515961 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tlx6\" (UniqueName: \"kubernetes.io/projected/e3b567aa-8ca1-4603-9e68-632b19ccceb6-kube-api-access-9tlx6\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:47.012328 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:47.012297 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:47.014674 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:47.014652 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls\") pod \"maas-api-e2e-api-b-619289-84dd47bd9b-kckxf\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:47.217438 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:47.217403 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:47.343827 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:47.343633 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf"] Jul 07 13:22:47.346666 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:22:47.346637 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b567aa_8ca1_4603_9e68_632b19ccceb6.slice/crio-8f020569178b68253077d91e709f4c81565730926d1db407b0e4fdbab01d246a WatchSource:0}: Error finding container 8f020569178b68253077d91e709f4c81565730926d1db407b0e4fdbab01d246a: Status 404 returned error can't find the container with id 8f020569178b68253077d91e709f4c81565730926d1db407b0e4fdbab01d246a Jul 07 13:22:48.254235 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:48.254201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" event={"ID":"e3b567aa-8ca1-4603-9e68-632b19ccceb6","Type":"ContainerStarted","Data":"cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8"} Jul 07 13:22:48.254443 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:48.254247 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" event={"ID":"e3b567aa-8ca1-4603-9e68-632b19ccceb6","Type":"ContainerStarted","Data":"8f020569178b68253077d91e709f4c81565730926d1db407b0e4fdbab01d246a"} Jul 07 13:22:48.254443 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:48.254314 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:48.274771 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:48.274687 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" podStartSLOduration=1.9179548450000001 podStartE2EDuration="2.274672231s" totalImagesPullingTime="356.717386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:48.274035361 +0000 UTC m=+2050.258883832" watchObservedRunningTime="2026-07-07 13:22:48.274672231 +0000 UTC m=+2050.259520704" Jul 07 13:22:54.264382 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:54.264351 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:22:59.577200 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.577168 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4"] Jul 07 13:22:59.577579 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.577383 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" podUID="af294b0d-aec7-4870-92d5-d0790c0fb026" containerName="maas-api" containerID="cri-o://d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090" gracePeriod=30 Jul 07 13:22:59.819553 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.819531 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:22:59.920524 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.920456 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtm4v\" (UniqueName: \"kubernetes.io/projected/af294b0d-aec7-4870-92d5-d0790c0fb026-kube-api-access-jtm4v\") pod \"af294b0d-aec7-4870-92d5-d0790c0fb026\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " Jul 07 13:22:59.920653 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.920534 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af294b0d-aec7-4870-92d5-d0790c0fb026-maas-api-tls\") pod \"af294b0d-aec7-4870-92d5-d0790c0fb026\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " Jul 07 13:22:59.920653 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.920578 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af294b0d-aec7-4870-92d5-d0790c0fb026-openshift-service-ca\") pod \"af294b0d-aec7-4870-92d5-d0790c0fb026\" (UID: \"af294b0d-aec7-4870-92d5-d0790c0fb026\") " Jul 07 13:22:59.920960 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.920930 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af294b0d-aec7-4870-92d5-d0790c0fb026-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af294b0d-aec7-4870-92d5-d0790c0fb026" (UID: "af294b0d-aec7-4870-92d5-d0790c0fb026"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:59.922517 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.922489 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af294b0d-aec7-4870-92d5-d0790c0fb026-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af294b0d-aec7-4870-92d5-d0790c0fb026" (UID: "af294b0d-aec7-4870-92d5-d0790c0fb026"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:22:59.922517 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:22:59.922499 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af294b0d-aec7-4870-92d5-d0790c0fb026-kube-api-access-jtm4v" (OuterVolumeSpecName: "kube-api-access-jtm4v") pod "af294b0d-aec7-4870-92d5-d0790c0fb026" (UID: "af294b0d-aec7-4870-92d5-d0790c0fb026"). InnerVolumeSpecName "kube-api-access-jtm4v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:23:00.022165 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.022096 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af294b0d-aec7-4870-92d5-d0790c0fb026-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:23:00.022165 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.022161 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtm4v\" (UniqueName: \"kubernetes.io/projected/af294b0d-aec7-4870-92d5-d0790c0fb026-kube-api-access-jtm4v\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:23:00.022165 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.022172 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af294b0d-aec7-4870-92d5-d0790c0fb026-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:23:00.308490 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.308454 2568 generic.go:356] "Generic (PLEG): container finished" podID="af294b0d-aec7-4870-92d5-d0790c0fb026" containerID="d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090" exitCode=0 Jul 07 13:23:00.308666 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.308513 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" Jul 07 13:23:00.308666 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.308536 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" event={"ID":"af294b0d-aec7-4870-92d5-d0790c0fb026","Type":"ContainerDied","Data":"d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090"} Jul 07 13:23:00.308666 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.308608 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4" event={"ID":"af294b0d-aec7-4870-92d5-d0790c0fb026","Type":"ContainerDied","Data":"cd0321a66216a1f24999d4b3b5d95286070120896ce961eb09462c3016acdbde"} Jul 07 13:23:00.308666 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.308642 2568 scope.go:117] "RemoveContainer" containerID="d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090" Jul 07 13:23:00.318396 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.318371 2568 scope.go:117] "RemoveContainer" containerID="d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090" Jul 07 13:23:00.318642 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:23:00.318626 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090\": container with ID starting with d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090 not found: ID does not exist" containerID="d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090" Jul 07 13:23:00.318694 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.318650 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090"} err="failed to get container status \"d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090\": rpc error: code = NotFound desc = could not find container \"d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090\": container with ID starting with d59fbaa76d604571ee948be05b9c5b08752180ddd7c4922cf3854d0789661090 not found: ID does not exist" Jul 07 13:23:00.331621 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.331599 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4"] Jul 07 13:23:00.335697 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.335678 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa07aa-746668b9d5-2s8b4"] Jul 07 13:23:00.602850 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:00.602770 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af294b0d-aec7-4870-92d5-d0790c0fb026" path="/var/lib/kubelet/pods/af294b0d-aec7-4870-92d5-d0790c0fb026/volumes" Jul 07 13:23:06.478208 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.478177 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf"] Jul 07 13:23:06.478682 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.478439 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" podUID="e3b567aa-8ca1-4603-9e68-632b19ccceb6" containerName="maas-api" containerID="cri-o://cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8" gracePeriod=30 Jul 07 13:23:06.721565 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.721535 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:23:06.777476 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.777404 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tlx6\" (UniqueName: \"kubernetes.io/projected/e3b567aa-8ca1-4603-9e68-632b19ccceb6-kube-api-access-9tlx6\") pod \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " Jul 07 13:23:06.777619 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.777512 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b567aa-8ca1-4603-9e68-632b19ccceb6-openshift-service-ca\") pod \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " Jul 07 13:23:06.777619 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.777563 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls\") pod \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\" (UID: \"e3b567aa-8ca1-4603-9e68-632b19ccceb6\") " Jul 07 13:23:06.777925 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.777891 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3b567aa-8ca1-4603-9e68-632b19ccceb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e3b567aa-8ca1-4603-9e68-632b19ccceb6" (UID: "e3b567aa-8ca1-4603-9e68-632b19ccceb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:23:06.779471 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.779445 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3b567aa-8ca1-4603-9e68-632b19ccceb6-kube-api-access-9tlx6" (OuterVolumeSpecName: "kube-api-access-9tlx6") pod "e3b567aa-8ca1-4603-9e68-632b19ccceb6" (UID: "e3b567aa-8ca1-4603-9e68-632b19ccceb6"). InnerVolumeSpecName "kube-api-access-9tlx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:23:06.779644 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.779626 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e3b567aa-8ca1-4603-9e68-632b19ccceb6" (UID: "e3b567aa-8ca1-4603-9e68-632b19ccceb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:23:06.878721 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.878696 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3b567aa-8ca1-4603-9e68-632b19ccceb6-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:23:06.878721 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.878720 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tlx6\" (UniqueName: \"kubernetes.io/projected/e3b567aa-8ca1-4603-9e68-632b19ccceb6-kube-api-access-9tlx6\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:23:06.878854 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:06.878731 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b567aa-8ca1-4603-9e68-632b19ccceb6-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:23:07.345132 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.345090 2568 generic.go:356] "Generic (PLEG): container finished" podID="e3b567aa-8ca1-4603-9e68-632b19ccceb6" containerID="cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8" exitCode=0 Jul 07 13:23:07.345310 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.345175 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" Jul 07 13:23:07.345310 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.345178 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" event={"ID":"e3b567aa-8ca1-4603-9e68-632b19ccceb6","Type":"ContainerDied","Data":"cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8"} Jul 07 13:23:07.345310 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.345267 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf" event={"ID":"e3b567aa-8ca1-4603-9e68-632b19ccceb6","Type":"ContainerDied","Data":"8f020569178b68253077d91e709f4c81565730926d1db407b0e4fdbab01d246a"} Jul 07 13:23:07.345310 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.345292 2568 scope.go:117] "RemoveContainer" containerID="cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8" Jul 07 13:23:07.354671 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.354655 2568 scope.go:117] "RemoveContainer" containerID="cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8" Jul 07 13:23:07.354922 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:23:07.354899 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8\": container with ID starting with cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8 not found: ID does not exist" containerID="cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8" Jul 07 13:23:07.354987 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.354930 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8"} err="failed to get container status \"cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8\": rpc error: code = NotFound desc = could not find container \"cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8\": container with ID starting with cdd71ca1ed557f41baa9ee23ea3698d1103ecedf9755db7401b2e16db16333f8 not found: ID does not exist" Jul 07 13:23:07.367020 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.366999 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf"] Jul 07 13:23:07.371411 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:07.371389 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-619289-84dd47bd9b-kckxf"] Jul 07 13:23:08.602580 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:08.602535 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3b567aa-8ca1-4603-9e68-632b19ccceb6" path="/var/lib/kubelet/pods/e3b567aa-8ca1-4603-9e68-632b19ccceb6/volumes" Jul 07 13:23:16.456237 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.456197 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8"] Jul 07 13:23:16.456697 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.456675 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af294b0d-aec7-4870-92d5-d0790c0fb026" containerName="maas-api" Jul 07 13:23:16.456697 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.456695 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="af294b0d-aec7-4870-92d5-d0790c0fb026" containerName="maas-api" Jul 07 13:23:16.456810 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.456715 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3b567aa-8ca1-4603-9e68-632b19ccceb6" containerName="maas-api" Jul 07 13:23:16.456810 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.456724 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3b567aa-8ca1-4603-9e68-632b19ccceb6" containerName="maas-api" Jul 07 13:23:16.456810 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.456804 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3b567aa-8ca1-4603-9e68-632b19ccceb6" containerName="maas-api" Jul 07 13:23:16.456959 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.456823 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="af294b0d-aec7-4870-92d5-d0790c0fb026" containerName="maas-api" Jul 07 13:23:16.459632 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.459611 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.468331 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.468313 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:23:16.468560 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.468544 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-951cca-serving-cert\"" Jul 07 13:23:16.480144 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.480106 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8"] Jul 07 13:23:16.552016 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.551991 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftvq5\" (UniqueName: \"kubernetes.io/projected/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-kube-api-access-ftvq5\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.552157 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.552036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-maas-api-tls\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.552157 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.552136 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-openshift-service-ca\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.653010 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.652985 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-maas-api-tls\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.653149 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.653032 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-openshift-service-ca\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.653149 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.653078 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftvq5\" (UniqueName: \"kubernetes.io/projected/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-kube-api-access-ftvq5\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.653717 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.653688 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-openshift-service-ca\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.655347 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.655330 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-maas-api-tls\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.666194 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.666173 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftvq5\" (UniqueName: \"kubernetes.io/projected/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-kube-api-access-ftvq5\") pod \"maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.769453 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.769425 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:16.907622 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:16.907597 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8"] Jul 07 13:23:16.909652 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:23:16.909619 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2682b17_fa32_40c9_ba9b_55a39aa9ba48.slice/crio-ffe17cad9410c5b296ec0075bc0fd88c0a3e8432ba994277be311dbed593deaf WatchSource:0}: Error finding container ffe17cad9410c5b296ec0075bc0fd88c0a3e8432ba994277be311dbed593deaf: Status 404 returned error can't find the container with id ffe17cad9410c5b296ec0075bc0fd88c0a3e8432ba994277be311dbed593deaf Jul 07 13:23:17.390096 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:17.390064 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" event={"ID":"c2682b17-fa32-40c9-ba9b-55a39aa9ba48","Type":"ContainerStarted","Data":"f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31"} Jul 07 13:23:17.390261 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:17.390126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" event={"ID":"c2682b17-fa32-40c9-ba9b-55a39aa9ba48","Type":"ContainerStarted","Data":"ffe17cad9410c5b296ec0075bc0fd88c0a3e8432ba994277be311dbed593deaf"} Jul 07 13:23:17.390261 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:17.390227 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:17.417556 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:17.417510 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" podStartSLOduration=1.064291549 podStartE2EDuration="1.417496579s" totalImagesPullingTime="353.20503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:23:17.416324642 +0000 UTC m=+2079.401173125" watchObservedRunningTime="2026-07-07 13:23:17.417496579 +0000 UTC m=+2079.402345050" Jul 07 13:23:23.400147 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:23.400102 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:23:24.455392 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.455355 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l"] Jul 07 13:23:24.458758 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.458741 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.462605 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.462583 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-38a70c-serving-cert\"" Jul 07 13:23:24.485856 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.485834 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l"] Jul 07 13:23:24.513618 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.513592 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa8f76e-be41-4a47-aa12-3c3184387edf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.513712 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.513644 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6aa8f76e-be41-4a47-aa12-3c3184387edf-maas-api-tls\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.513712 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.513691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtdbn\" (UniqueName: \"kubernetes.io/projected/6aa8f76e-be41-4a47-aa12-3c3184387edf-kube-api-access-jtdbn\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.614640 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.614607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa8f76e-be41-4a47-aa12-3c3184387edf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.614794 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.614699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6aa8f76e-be41-4a47-aa12-3c3184387edf-maas-api-tls\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.614794 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.614770 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtdbn\" (UniqueName: \"kubernetes.io/projected/6aa8f76e-be41-4a47-aa12-3c3184387edf-kube-api-access-jtdbn\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.615303 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.615283 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa8f76e-be41-4a47-aa12-3c3184387edf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.617164 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.617142 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6aa8f76e-be41-4a47-aa12-3c3184387edf-maas-api-tls\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.623853 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.623787 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtdbn\" (UniqueName: \"kubernetes.io/projected/6aa8f76e-be41-4a47-aa12-3c3184387edf-kube-api-access-jtdbn\") pod \"maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.769091 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.769061 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:24.914628 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:24.914593 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l"] Jul 07 13:23:24.922558 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:23:24.922528 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa8f76e_be41_4a47_aa12_3c3184387edf.slice/crio-c61cd60c2f8a61736b1c1bab8095103e076fcb2185cd3b68384ef0ba1e6a36be WatchSource:0}: Error finding container c61cd60c2f8a61736b1c1bab8095103e076fcb2185cd3b68384ef0ba1e6a36be: Status 404 returned error can't find the container with id c61cd60c2f8a61736b1c1bab8095103e076fcb2185cd3b68384ef0ba1e6a36be Jul 07 13:23:25.435163 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:25.435126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" event={"ID":"6aa8f76e-be41-4a47-aa12-3c3184387edf","Type":"ContainerStarted","Data":"7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3"} Jul 07 13:23:25.435364 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:25.435199 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" event={"ID":"6aa8f76e-be41-4a47-aa12-3c3184387edf","Type":"ContainerStarted","Data":"c61cd60c2f8a61736b1c1bab8095103e076fcb2185cd3b68384ef0ba1e6a36be"} Jul 07 13:23:25.435364 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:25.435233 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:23:25.452137 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:25.452073 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" podStartSLOduration=1.08437775 podStartE2EDuration="1.452059619s" totalImagesPullingTime="367.681869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:23:25.451290183 +0000 UTC m=+2087.436138657" watchObservedRunningTime="2026-07-07 13:23:25.452059619 +0000 UTC m=+2087.436908091" Jul 07 13:23:31.445203 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:23:31.445170 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:24:13.107653 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.107622 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9"] Jul 07 13:24:13.110845 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.110823 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.113818 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.113798 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-38a70c\"/\"default-dockercfg-qz2q2\"" Jul 07 13:24:13.113929 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.113867 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-38a70c\"/\"openshift-service-ca.crt\"" Jul 07 13:24:13.115233 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.115213 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-38a70c\"/\"auth-test-model-38a70c-kserve-self-signed-certs\"" Jul 07 13:24:13.115233 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.115227 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-38a70c\"/\"kube-root-ca.crt\"" Jul 07 13:24:13.123340 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.123317 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9"] Jul 07 13:24:13.206860 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.206835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-home\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.206984 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.206877 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tls-certs\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.206984 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.206936 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl7wp\" (UniqueName: \"kubernetes.io/projected/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-kube-api-access-bl7wp\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.206984 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.206960 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-dshm\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.207090 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.207023 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-model-cache\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.207090 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.207064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tmp-dir\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308092 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tls-certs\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308253 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl7wp\" (UniqueName: \"kubernetes.io/projected/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-kube-api-access-bl7wp\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308253 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-dshm\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308253 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-model-cache\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308253 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308211 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tmp-dir\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308253 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308231 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-home\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308598 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308578 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-model-cache\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308679 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308632 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-home\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.308679 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.308661 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tmp-dir\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.310809 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.310783 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-dshm\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.311628 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.311603 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tls-certs\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.316705 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.316685 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl7wp\" (UniqueName: \"kubernetes.io/projected/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-kube-api-access-bl7wp\") pod \"auth-test-model-38a70c-kserve-54b4589dc5-rlxm9\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.420133 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.420045 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:13.549837 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.549805 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9"] Jul 07 13:24:13.552726 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:24:13.552701 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c44c1f6_880b_4c3d_94c1_eac89dfa6337.slice/crio-746551bdfd56504f527ab50804b10df456b840a1d0cf860eb08da31b4c2fdce4 WatchSource:0}: Error finding container 746551bdfd56504f527ab50804b10df456b840a1d0cf860eb08da31b4c2fdce4: Status 404 returned error can't find the container with id 746551bdfd56504f527ab50804b10df456b840a1d0cf860eb08da31b4c2fdce4 Jul 07 13:24:13.644991 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:13.644964 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" event={"ID":"2c44c1f6-880b-4c3d-94c1-eac89dfa6337","Type":"ContainerStarted","Data":"746551bdfd56504f527ab50804b10df456b840a1d0cf860eb08da31b4c2fdce4"} Jul 07 13:24:15.657440 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:15.657405 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" event={"ID":"2c44c1f6-880b-4c3d-94c1-eac89dfa6337","Type":"ContainerStarted","Data":"b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153"} Jul 07 13:24:15.680404 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:15.680350 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" podStartSLOduration=1.493001379 podStartE2EDuration="2.680333112s" totalImagesPullingTime="1.187331733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:24:15.680173582 +0000 UTC m=+2137.665022054" watchObservedRunningTime="2026-07-07 13:24:15.680333112 +0000 UTC m=+2137.665181585" Jul 07 13:24:23.420893 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:23.420861 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:23.421297 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:23.420908 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:23.433206 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:23.433183 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:24:23.703876 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:24:23.703798 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:26:18.703713 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:18.703637 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9"] Jul 07 13:26:33.723987 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:33.723929 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" podUID="2c44c1f6-880b-4c3d-94c1-eac89dfa6337" containerName="main" containerID="cri-o://b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153" gracePeriod=45 Jul 07 13:26:33.972413 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:33.972389 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:26:34.080166 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080070 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-home\") pod \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " Jul 07 13:26:34.080166 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080106 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-dshm\") pod \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " Jul 07 13:26:34.080339 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080211 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-model-cache\") pod \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " Jul 07 13:26:34.080339 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080274 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tls-certs\") pod \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " Jul 07 13:26:34.080339 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080305 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl7wp\" (UniqueName: \"kubernetes.io/projected/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-kube-api-access-bl7wp\") pod \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " Jul 07 13:26:34.080490 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080347 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tmp-dir\") pod \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\" (UID: \"2c44c1f6-880b-4c3d-94c1-eac89dfa6337\") " Jul 07 13:26:34.080490 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080377 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-home" (OuterVolumeSpecName: "home") pod "2c44c1f6-880b-4c3d-94c1-eac89dfa6337" (UID: "2c44c1f6-880b-4c3d-94c1-eac89dfa6337"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:26:34.080599 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080540 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c44c1f6-880b-4c3d-94c1-eac89dfa6337" (UID: "2c44c1f6-880b-4c3d-94c1-eac89dfa6337"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:26:34.080654 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080606 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-home\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:26:34.080712 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.080653 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c44c1f6-880b-4c3d-94c1-eac89dfa6337" (UID: "2c44c1f6-880b-4c3d-94c1-eac89dfa6337"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:26:34.082432 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.082404 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-dshm" (OuterVolumeSpecName: "dshm") pod "2c44c1f6-880b-4c3d-94c1-eac89dfa6337" (UID: "2c44c1f6-880b-4c3d-94c1-eac89dfa6337"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:26:34.082565 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.082541 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c44c1f6-880b-4c3d-94c1-eac89dfa6337" (UID: "2c44c1f6-880b-4c3d-94c1-eac89dfa6337"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:26:34.082612 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.082550 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-kube-api-access-bl7wp" (OuterVolumeSpecName: "kube-api-access-bl7wp") pod "2c44c1f6-880b-4c3d-94c1-eac89dfa6337" (UID: "2c44c1f6-880b-4c3d-94c1-eac89dfa6337"). InnerVolumeSpecName "kube-api-access-bl7wp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:26:34.181181 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.181148 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl7wp\" (UniqueName: \"kubernetes.io/projected/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-kube-api-access-bl7wp\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:26:34.181181 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.181176 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tmp-dir\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:26:34.181181 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.181186 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-dshm\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:26:34.181410 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.181193 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-model-cache\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:26:34.181410 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.181202 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c44c1f6-880b-4c3d-94c1-eac89dfa6337-tls-certs\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:26:34.264849 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.264817 2568 generic.go:356] "Generic (PLEG): container finished" podID="2c44c1f6-880b-4c3d-94c1-eac89dfa6337" containerID="b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153" exitCode=0 Jul 07 13:26:34.265020 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.264856 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" Jul 07 13:26:34.265020 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.264904 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" event={"ID":"2c44c1f6-880b-4c3d-94c1-eac89dfa6337","Type":"ContainerDied","Data":"b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153"} Jul 07 13:26:34.265020 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.264945 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9" event={"ID":"2c44c1f6-880b-4c3d-94c1-eac89dfa6337","Type":"ContainerDied","Data":"746551bdfd56504f527ab50804b10df456b840a1d0cf860eb08da31b4c2fdce4"} Jul 07 13:26:34.265020 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.264963 2568 scope.go:117] "RemoveContainer" containerID="b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153" Jul 07 13:26:34.273803 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.273787 2568 scope.go:117] "RemoveContainer" containerID="b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153" Jul 07 13:26:34.274050 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:26:34.274032 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153\": container with ID starting with b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153 not found: ID does not exist" containerID="b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153" Jul 07 13:26:34.274148 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.274059 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153"} err="failed to get container status \"b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153\": rpc error: code = NotFound desc = could not find container \"b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153\": container with ID starting with b5de3efa5c770f9beb022cd136ecba278f6eb0b6d0ff46cdee93dfbfc101e153 not found: ID does not exist" Jul 07 13:26:34.286626 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.286602 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9"] Jul 07 13:26:34.289133 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.289097 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-38a70c/auth-test-model-38a70c-kserve-54b4589dc5-rlxm9"] Jul 07 13:26:34.602574 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:34.602534 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c44c1f6-880b-4c3d-94c1-eac89dfa6337" path="/var/lib/kubelet/pods/2c44c1f6-880b-4c3d-94c1-eac89dfa6337/volumes" Jul 07 13:26:35.495998 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.495966 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98"] Jul 07 13:26:35.496364 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.496353 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c44c1f6-880b-4c3d-94c1-eac89dfa6337" containerName="main" Jul 07 13:26:35.496406 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.496366 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c44c1f6-880b-4c3d-94c1-eac89dfa6337" containerName="main" Jul 07 13:26:35.496441 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.496421 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c44c1f6-880b-4c3d-94c1-eac89dfa6337" containerName="main" Jul 07 13:26:35.500707 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.500691 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.503315 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.503289 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-951cca\"/\"kube-root-ca.crt\"" Jul 07 13:26:35.504144 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.504102 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-951cca\"/\"openshift-service-ca.crt\"" Jul 07 13:26:35.504269 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.504249 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-951cca\"/\"default-dockercfg-7fh7f\"" Jul 07 13:26:35.504324 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.504289 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-951cca\"/\"sub-test-model-951cca-kserve-self-signed-certs\"" Jul 07 13:26:35.508356 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.508333 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98"] Jul 07 13:26:35.593067 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.593038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-tmp-dir\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.593218 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.593079 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-model-cache\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.593218 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.593102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-home\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.593218 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.593198 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-dshm\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.593329 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.593225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63bd50f8-780a-4f66-acd1-5c4575da3800-tls-certs\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.593329 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.593261 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79h87\" (UniqueName: \"kubernetes.io/projected/63bd50f8-780a-4f66-acd1-5c4575da3800-kube-api-access-79h87\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.694450 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.694410 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-model-cache\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.694450 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.694452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-home\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.694706 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.694488 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-dshm\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.694706 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.694502 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63bd50f8-780a-4f66-acd1-5c4575da3800-tls-certs\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.694706 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.694536 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79h87\" (UniqueName: \"kubernetes.io/projected/63bd50f8-780a-4f66-acd1-5c4575da3800-kube-api-access-79h87\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.694706 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.694629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-tmp-dir\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.694914 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.694875 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-model-cache\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.694977 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.694935 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-home\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.695061 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.695039 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-tmp-dir\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.696850 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.696828 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-dshm\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.697026 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.697010 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63bd50f8-780a-4f66-acd1-5c4575da3800-tls-certs\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.704169 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.704105 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79h87\" (UniqueName: \"kubernetes.io/projected/63bd50f8-780a-4f66-acd1-5c4575da3800-kube-api-access-79h87\") pod \"sub-test-model-951cca-kserve-6dbdc99595-cks98\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.810907 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.810824 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:35.937106 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:35.937075 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98"] Jul 07 13:26:35.939422 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:26:35.939397 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63bd50f8_780a_4f66_acd1_5c4575da3800.slice/crio-1e0f460ca506fdc0fdfc7148c3f1de03170fc79816ec5515f5d64315c2dc5366 WatchSource:0}: Error finding container 1e0f460ca506fdc0fdfc7148c3f1de03170fc79816ec5515f5d64315c2dc5366: Status 404 returned error can't find the container with id 1e0f460ca506fdc0fdfc7148c3f1de03170fc79816ec5515f5d64315c2dc5366 Jul 07 13:26:36.276210 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:36.276172 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" event={"ID":"63bd50f8-780a-4f66-acd1-5c4575da3800","Type":"ContainerStarted","Data":"b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90"} Jul 07 13:26:36.276397 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:36.276220 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" event={"ID":"63bd50f8-780a-4f66-acd1-5c4575da3800","Type":"ContainerStarted","Data":"1e0f460ca506fdc0fdfc7148c3f1de03170fc79816ec5515f5d64315c2dc5366"} Jul 07 13:26:36.293462 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:36.293413 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" podStartSLOduration=1.2933987789999999 podStartE2EDuration="1.293398779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:26:36.292630016 +0000 UTC m=+2278.277478489" watchObservedRunningTime="2026-07-07 13:26:36.293398779 +0000 UTC m=+2278.278247251" Jul 07 13:26:45.811197 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:45.811161 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:45.811587 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:45.811234 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:45.823336 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:45.823314 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:26:46.330008 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:26:46.329975 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:27:42.922661 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:42.922588 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98"] Jul 07 13:27:53.909091 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:53.909058 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl"] Jul 07 13:27:53.912325 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:53.912298 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:53.915325 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:53.915300 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-951cca\"/\"rate-test-model-951cca-kserve-self-signed-certs\"" Jul 07 13:27:53.925238 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:53.925203 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl"] Jul 07 13:27:54.032726 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.032681 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-home\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.032726 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.032732 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-dshm\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.032943 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.032834 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-model-cache\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.032943 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.032869 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tls-certs\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.032943 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.032899 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfl64\" (UniqueName: \"kubernetes.io/projected/099e9658-aedb-427c-a8e9-cae9aad0e6ce-kube-api-access-sfl64\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.032943 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.032928 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tmp-dir\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.134323 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.134275 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-home\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.134323 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.134331 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-dshm\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.134586 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.134382 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-model-cache\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.134586 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.134404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tls-certs\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.134586 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.134555 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfl64\" (UniqueName: \"kubernetes.io/projected/099e9658-aedb-427c-a8e9-cae9aad0e6ce-kube-api-access-sfl64\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.134747 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.134606 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tmp-dir\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.134747 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.134702 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-home\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.134999 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.134975 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-model-cache\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.135090 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.135054 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tmp-dir\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.136740 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.136714 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-dshm\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.137029 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.137010 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tls-certs\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.142655 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.142636 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfl64\" (UniqueName: \"kubernetes.io/projected/099e9658-aedb-427c-a8e9-cae9aad0e6ce-kube-api-access-sfl64\") pod \"rate-test-model-951cca-kserve-795bdfb5c9-f2gnl\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.222855 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.222830 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:27:54.347983 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.347957 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl"] Jul 07 13:27:54.349949 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:27:54.349915 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod099e9658_aedb_427c_a8e9_cae9aad0e6ce.slice/crio-aa94f6644ae0785e6d22fbe437f697e2480a0c646b39449dc7cc5a052b2ebe1f WatchSource:0}: Error finding container aa94f6644ae0785e6d22fbe437f697e2480a0c646b39449dc7cc5a052b2ebe1f: Status 404 returned error can't find the container with id aa94f6644ae0785e6d22fbe437f697e2480a0c646b39449dc7cc5a052b2ebe1f Jul 07 13:27:54.621503 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.621418 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" event={"ID":"099e9658-aedb-427c-a8e9-cae9aad0e6ce","Type":"ContainerStarted","Data":"2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96"} Jul 07 13:27:54.621503 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.621453 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" event={"ID":"099e9658-aedb-427c-a8e9-cae9aad0e6ce","Type":"ContainerStarted","Data":"aa94f6644ae0785e6d22fbe437f697e2480a0c646b39449dc7cc5a052b2ebe1f"} Jul 07 13:27:54.639383 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:54.639337 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" podStartSLOduration=1.639324796 podStartE2EDuration="1.639324796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:27:54.639309133 +0000 UTC m=+2356.624157605" watchObservedRunningTime="2026-07-07 13:27:54.639324796 +0000 UTC m=+2356.624173267" Jul 07 13:27:57.942636 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:57.942590 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" podUID="63bd50f8-780a-4f66-acd1-5c4575da3800" containerName="main" containerID="cri-o://b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90" gracePeriod=45 Jul 07 13:27:58.186312 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.186286 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:27:58.373093 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373063 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63bd50f8-780a-4f66-acd1-5c4575da3800-tls-certs\") pod \"63bd50f8-780a-4f66-acd1-5c4575da3800\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " Jul 07 13:27:58.373304 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373166 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-dshm\") pod \"63bd50f8-780a-4f66-acd1-5c4575da3800\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " Jul 07 13:27:58.373304 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373214 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-model-cache\") pod \"63bd50f8-780a-4f66-acd1-5c4575da3800\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " Jul 07 13:27:58.373304 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373234 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-home\") pod \"63bd50f8-780a-4f66-acd1-5c4575da3800\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " Jul 07 13:27:58.373304 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373258 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79h87\" (UniqueName: \"kubernetes.io/projected/63bd50f8-780a-4f66-acd1-5c4575da3800-kube-api-access-79h87\") pod \"63bd50f8-780a-4f66-acd1-5c4575da3800\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " Jul 07 13:27:58.373304 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373288 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-tmp-dir\") pod \"63bd50f8-780a-4f66-acd1-5c4575da3800\" (UID: \"63bd50f8-780a-4f66-acd1-5c4575da3800\") " Jul 07 13:27:58.373579 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373480 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-model-cache" (OuterVolumeSpecName: "model-cache") pod "63bd50f8-780a-4f66-acd1-5c4575da3800" (UID: "63bd50f8-780a-4f66-acd1-5c4575da3800"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:27:58.373579 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373492 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-home" (OuterVolumeSpecName: "home") pod "63bd50f8-780a-4f66-acd1-5c4575da3800" (UID: "63bd50f8-780a-4f66-acd1-5c4575da3800"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:27:58.373579 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373561 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-model-cache\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:27:58.373725 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373581 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-home\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:27:58.373725 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.373618 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63bd50f8-780a-4f66-acd1-5c4575da3800" (UID: "63bd50f8-780a-4f66-acd1-5c4575da3800"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:27:58.375468 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.375439 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63bd50f8-780a-4f66-acd1-5c4575da3800-kube-api-access-79h87" (OuterVolumeSpecName: "kube-api-access-79h87") pod "63bd50f8-780a-4f66-acd1-5c4575da3800" (UID: "63bd50f8-780a-4f66-acd1-5c4575da3800"). InnerVolumeSpecName "kube-api-access-79h87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:27:58.375468 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.375458 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-dshm" (OuterVolumeSpecName: "dshm") pod "63bd50f8-780a-4f66-acd1-5c4575da3800" (UID: "63bd50f8-780a-4f66-acd1-5c4575da3800"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:27:58.375468 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.375464 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63bd50f8-780a-4f66-acd1-5c4575da3800-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63bd50f8-780a-4f66-acd1-5c4575da3800" (UID: "63bd50f8-780a-4f66-acd1-5c4575da3800"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:27:58.474652 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.474627 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-dshm\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:27:58.474652 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.474651 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79h87\" (UniqueName: \"kubernetes.io/projected/63bd50f8-780a-4f66-acd1-5c4575da3800-kube-api-access-79h87\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:27:58.474890 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.474662 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63bd50f8-780a-4f66-acd1-5c4575da3800-tmp-dir\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:27:58.474890 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.474670 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63bd50f8-780a-4f66-acd1-5c4575da3800-tls-certs\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:27:58.638771 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.638706 2568 generic.go:356] "Generic (PLEG): container finished" podID="63bd50f8-780a-4f66-acd1-5c4575da3800" containerID="b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90" exitCode=0 Jul 07 13:27:58.638771 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.638755 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" Jul 07 13:27:58.638907 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.638777 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" event={"ID":"63bd50f8-780a-4f66-acd1-5c4575da3800","Type":"ContainerDied","Data":"b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90"} Jul 07 13:27:58.638907 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.638811 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98" event={"ID":"63bd50f8-780a-4f66-acd1-5c4575da3800","Type":"ContainerDied","Data":"1e0f460ca506fdc0fdfc7148c3f1de03170fc79816ec5515f5d64315c2dc5366"} Jul 07 13:27:58.638907 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.638833 2568 scope.go:117] "RemoveContainer" containerID="b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90" Jul 07 13:27:58.648696 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.648681 2568 scope.go:117] "RemoveContainer" containerID="b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90" Jul 07 13:27:58.649349 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:27:58.649328 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90\": container with ID starting with b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90 not found: ID does not exist" containerID="b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90" Jul 07 13:27:58.649443 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.649355 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90"} err="failed to get container status \"b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90\": rpc error: code = NotFound desc = could not find container \"b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90\": container with ID starting with b22f04ac72a61f8fa32a142f797846c701281d289374f0b084fdff01c07adf90 not found: ID does not exist" Jul 07 13:27:58.656401 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.656382 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98"] Jul 07 13:27:58.657854 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:27:58.657834 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/sub-test-model-951cca-kserve-6dbdc99595-cks98"] Jul 07 13:28:00.602535 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:28:00.602494 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63bd50f8-780a-4f66-acd1-5c4575da3800" path="/var/lib/kubelet/pods/63bd50f8-780a-4f66-acd1-5c4575da3800/volumes" Jul 07 13:28:04.223362 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:28:04.223309 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:28:04.223753 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:28:04.223378 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:28:04.235635 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:28:04.235607 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:28:04.676595 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:28:04.676507 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:29:03.073015 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:03.072942 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl"] Jul 07 13:29:18.092768 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.092728 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" podUID="099e9658-aedb-427c-a8e9-cae9aad0e6ce" containerName="main" containerID="cri-o://2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96" gracePeriod=45 Jul 07 13:29:18.337214 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.337187 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:29:18.448830 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.448760 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-dshm\") pod \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " Jul 07 13:29:18.448830 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.448815 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-home\") pod \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " Jul 07 13:29:18.448995 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.448846 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfl64\" (UniqueName: \"kubernetes.io/projected/099e9658-aedb-427c-a8e9-cae9aad0e6ce-kube-api-access-sfl64\") pod \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " Jul 07 13:29:18.448995 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.448863 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tmp-dir\") pod \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " Jul 07 13:29:18.448995 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.448889 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tls-certs\") pod \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " Jul 07 13:29:18.448995 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.448919 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-model-cache\") pod \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\" (UID: \"099e9658-aedb-427c-a8e9-cae9aad0e6ce\") " Jul 07 13:29:18.449239 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.449192 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "099e9658-aedb-427c-a8e9-cae9aad0e6ce" (UID: "099e9658-aedb-427c-a8e9-cae9aad0e6ce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:29:18.449307 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.449211 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-home" (OuterVolumeSpecName: "home") pod "099e9658-aedb-427c-a8e9-cae9aad0e6ce" (UID: "099e9658-aedb-427c-a8e9-cae9aad0e6ce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:29:18.449365 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.449301 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-model-cache" (OuterVolumeSpecName: "model-cache") pod "099e9658-aedb-427c-a8e9-cae9aad0e6ce" (UID: "099e9658-aedb-427c-a8e9-cae9aad0e6ce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:29:18.450900 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.450878 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-dshm" (OuterVolumeSpecName: "dshm") pod "099e9658-aedb-427c-a8e9-cae9aad0e6ce" (UID: "099e9658-aedb-427c-a8e9-cae9aad0e6ce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:29:18.451011 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.450913 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "099e9658-aedb-427c-a8e9-cae9aad0e6ce" (UID: "099e9658-aedb-427c-a8e9-cae9aad0e6ce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:29:18.451011 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.450942 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099e9658-aedb-427c-a8e9-cae9aad0e6ce-kube-api-access-sfl64" (OuterVolumeSpecName: "kube-api-access-sfl64") pod "099e9658-aedb-427c-a8e9-cae9aad0e6ce" (UID: "099e9658-aedb-427c-a8e9-cae9aad0e6ce"). InnerVolumeSpecName "kube-api-access-sfl64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:29:18.550263 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.550238 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-dshm\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:18.550263 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.550260 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-home\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:18.550418 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.550273 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfl64\" (UniqueName: \"kubernetes.io/projected/099e9658-aedb-427c-a8e9-cae9aad0e6ce-kube-api-access-sfl64\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:18.550418 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.550290 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tmp-dir\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:18.550418 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.550302 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/099e9658-aedb-427c-a8e9-cae9aad0e6ce-tls-certs\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:18.550418 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.550314 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/099e9658-aedb-427c-a8e9-cae9aad0e6ce-model-cache\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:18.999544 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.999510 2568 generic.go:356] "Generic (PLEG): container finished" podID="099e9658-aedb-427c-a8e9-cae9aad0e6ce" containerID="2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96" exitCode=0 Jul 07 13:29:18.999544 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.999548 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" Jul 07 13:29:18.999754 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.999589 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" event={"ID":"099e9658-aedb-427c-a8e9-cae9aad0e6ce","Type":"ContainerDied","Data":"2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96"} Jul 07 13:29:18.999754 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.999631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl" event={"ID":"099e9658-aedb-427c-a8e9-cae9aad0e6ce","Type":"ContainerDied","Data":"aa94f6644ae0785e6d22fbe437f697e2480a0c646b39449dc7cc5a052b2ebe1f"} Jul 07 13:29:18.999754 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:18.999648 2568 scope.go:117] "RemoveContainer" containerID="2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96" Jul 07 13:29:19.008460 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:19.008442 2568 scope.go:117] "RemoveContainer" containerID="2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96" Jul 07 13:29:19.008702 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:29:19.008679 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96\": container with ID starting with 2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96 not found: ID does not exist" containerID="2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96" Jul 07 13:29:19.008747 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:19.008712 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96"} err="failed to get container status \"2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96\": rpc error: code = NotFound desc = could not find container \"2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96\": container with ID starting with 2e27bc15e0326f67757ce2a90fc0de5643fe82385fdd91c2850face443e4db96 not found: ID does not exist" Jul 07 13:29:19.016203 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:19.016181 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl"] Jul 07 13:29:19.019920 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:19.019897 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-951cca/rate-test-model-951cca-kserve-795bdfb5c9-f2gnl"] Jul 07 13:29:20.603305 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:20.603269 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099e9658-aedb-427c-a8e9-cae9aad0e6ce" path="/var/lib/kubelet/pods/099e9658-aedb-427c-a8e9-cae9aad0e6ce/volumes" Jul 07 13:29:47.969358 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:47.969326 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8"] Jul 07 13:29:47.969844 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:47.969563 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" podUID="c2682b17-fa32-40c9-ba9b-55a39aa9ba48" containerName="maas-api" containerID="cri-o://f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31" gracePeriod=30 Jul 07 13:29:48.409073 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.409049 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:29:48.594883 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.594793 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftvq5\" (UniqueName: \"kubernetes.io/projected/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-kube-api-access-ftvq5\") pod \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " Jul 07 13:29:48.594883 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.594830 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-openshift-service-ca\") pod \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " Jul 07 13:29:48.594883 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.594880 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-maas-api-tls\") pod \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\" (UID: \"c2682b17-fa32-40c9-ba9b-55a39aa9ba48\") " Jul 07 13:29:48.595277 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.595250 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c2682b17-fa32-40c9-ba9b-55a39aa9ba48" (UID: "c2682b17-fa32-40c9-ba9b-55a39aa9ba48"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:29:48.596913 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.596893 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-kube-api-access-ftvq5" (OuterVolumeSpecName: "kube-api-access-ftvq5") pod "c2682b17-fa32-40c9-ba9b-55a39aa9ba48" (UID: "c2682b17-fa32-40c9-ba9b-55a39aa9ba48"). InnerVolumeSpecName "kube-api-access-ftvq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:29:48.597004 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.596979 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c2682b17-fa32-40c9-ba9b-55a39aa9ba48" (UID: "c2682b17-fa32-40c9-ba9b-55a39aa9ba48"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:29:48.696096 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.696074 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftvq5\" (UniqueName: \"kubernetes.io/projected/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-kube-api-access-ftvq5\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:48.696096 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.696097 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:48.696253 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:48.696107 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2682b17-fa32-40c9-ba9b-55a39aa9ba48-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:49.144242 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.144204 2568 generic.go:356] "Generic (PLEG): container finished" podID="c2682b17-fa32-40c9-ba9b-55a39aa9ba48" containerID="f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31" exitCode=0 Jul 07 13:29:49.144671 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.144253 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" Jul 07 13:29:49.144671 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.144285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" event={"ID":"c2682b17-fa32-40c9-ba9b-55a39aa9ba48","Type":"ContainerDied","Data":"f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31"} Jul 07 13:29:49.144671 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.144336 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" event={"ID":"c2682b17-fa32-40c9-ba9b-55a39aa9ba48","Type":"ContainerDied","Data":"ffe17cad9410c5b296ec0075bc0fd88c0a3e8432ba994277be311dbed593deaf"} Jul 07 13:29:49.144671 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.144358 2568 scope.go:117] "RemoveContainer" containerID="f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31" Jul 07 13:29:49.152633 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.152614 2568 scope.go:117] "RemoveContainer" containerID="f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31" Jul 07 13:29:49.152883 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:29:49.152864 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31\": container with ID starting with f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31 not found: ID does not exist" containerID="f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31" Jul 07 13:29:49.152938 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.152892 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31"} err="failed to get container status \"f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31\": rpc error: code = NotFound desc = could not find container \"f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31\": container with ID starting with f7e2d389f4b011fcb5bed713d44cd28a6a0817bd5aa8696e79bc0cc42fc28d31 not found: ID does not exist" Jul 07 13:29:49.165977 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.165956 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8"] Jul 07 13:29:49.169982 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:49.169962 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8"] Jul 07 13:29:50.603313 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:50.603275 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2682b17-fa32-40c9-ba9b-55a39aa9ba48" path="/var/lib/kubelet/pods/c2682b17-fa32-40c9-ba9b-55a39aa9ba48/volumes" Jul 07 13:29:51.395175 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:51.395131 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-a-951cca-86859b7c55-2nxh8" podUID="c2682b17-fa32-40c9-ba9b-55a39aa9ba48" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.33:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 07 13:29:55.593467 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.593434 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l"] Jul 07 13:29:55.593912 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.593698 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" podUID="6aa8f76e-be41-4a47-aa12-3c3184387edf" containerName="maas-api" containerID="cri-o://7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3" gracePeriod=30 Jul 07 13:29:55.837279 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.837253 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:29:55.846825 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.846781 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa8f76e-be41-4a47-aa12-3c3184387edf-openshift-service-ca\") pod \"6aa8f76e-be41-4a47-aa12-3c3184387edf\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " Jul 07 13:29:55.846825 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.846817 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6aa8f76e-be41-4a47-aa12-3c3184387edf-maas-api-tls\") pod \"6aa8f76e-be41-4a47-aa12-3c3184387edf\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " Jul 07 13:29:55.846985 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.846880 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtdbn\" (UniqueName: \"kubernetes.io/projected/6aa8f76e-be41-4a47-aa12-3c3184387edf-kube-api-access-jtdbn\") pod \"6aa8f76e-be41-4a47-aa12-3c3184387edf\" (UID: \"6aa8f76e-be41-4a47-aa12-3c3184387edf\") " Jul 07 13:29:55.847188 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.847164 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aa8f76e-be41-4a47-aa12-3c3184387edf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6aa8f76e-be41-4a47-aa12-3c3184387edf" (UID: "6aa8f76e-be41-4a47-aa12-3c3184387edf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:29:55.848743 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.848723 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6aa8f76e-be41-4a47-aa12-3c3184387edf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6aa8f76e-be41-4a47-aa12-3c3184387edf" (UID: "6aa8f76e-be41-4a47-aa12-3c3184387edf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:29:55.848889 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.848869 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aa8f76e-be41-4a47-aa12-3c3184387edf-kube-api-access-jtdbn" (OuterVolumeSpecName: "kube-api-access-jtdbn") pod "6aa8f76e-be41-4a47-aa12-3c3184387edf" (UID: "6aa8f76e-be41-4a47-aa12-3c3184387edf"). InnerVolumeSpecName "kube-api-access-jtdbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:29:55.947949 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.947924 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtdbn\" (UniqueName: \"kubernetes.io/projected/6aa8f76e-be41-4a47-aa12-3c3184387edf-kube-api-access-jtdbn\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:55.947949 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.947947 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa8f76e-be41-4a47-aa12-3c3184387edf-openshift-service-ca\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:55.948093 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:55.947958 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6aa8f76e-be41-4a47-aa12-3c3184387edf-maas-api-tls\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:29:56.176594 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.176521 2568 generic.go:356] "Generic (PLEG): container finished" podID="6aa8f76e-be41-4a47-aa12-3c3184387edf" containerID="7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3" exitCode=0 Jul 07 13:29:56.176594 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.176562 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" Jul 07 13:29:56.176747 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.176611 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" event={"ID":"6aa8f76e-be41-4a47-aa12-3c3184387edf","Type":"ContainerDied","Data":"7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3"} Jul 07 13:29:56.176747 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.176660 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l" event={"ID":"6aa8f76e-be41-4a47-aa12-3c3184387edf","Type":"ContainerDied","Data":"c61cd60c2f8a61736b1c1bab8095103e076fcb2185cd3b68384ef0ba1e6a36be"} Jul 07 13:29:56.176747 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.176682 2568 scope.go:117] "RemoveContainer" containerID="7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3" Jul 07 13:29:56.185783 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.185764 2568 scope.go:117] "RemoveContainer" containerID="7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3" Jul 07 13:29:56.186017 ip-10-0-131-17 kubenswrapper[2568]: E0707 13:29:56.186000 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3\": container with ID starting with 7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3 not found: ID does not exist" containerID="7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3" Jul 07 13:29:56.186066 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.186025 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3"} err="failed to get container status \"7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3\": rpc error: code = NotFound desc = could not find container \"7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3\": container with ID starting with 7a83661e42cfc75f95942cc2e249723169649dc80c8e9af8fad4629b2f8a10a3 not found: ID does not exist" Jul 07 13:29:56.198612 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.198590 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l"] Jul 07 13:29:56.204473 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.204454 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-38a70c-75ff58d55d-h9l7l"] Jul 07 13:29:56.603632 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:29:56.603597 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aa8f76e-be41-4a47-aa12-3c3184387edf" path="/var/lib/kubelet/pods/6aa8f76e-be41-4a47-aa12-3c3184387edf/volumes" Jul 07 13:30:00.141152 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141102 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-dhtzz"] Jul 07 13:30:00.141530 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141475 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aa8f76e-be41-4a47-aa12-3c3184387edf" containerName="maas-api" Jul 07 13:30:00.141530 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141488 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aa8f76e-be41-4a47-aa12-3c3184387edf" containerName="maas-api" Jul 07 13:30:00.141530 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141505 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63bd50f8-780a-4f66-acd1-5c4575da3800" containerName="main" Jul 07 13:30:00.141530 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141513 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="63bd50f8-780a-4f66-acd1-5c4575da3800" containerName="main" Jul 07 13:30:00.141530 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141525 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2682b17-fa32-40c9-ba9b-55a39aa9ba48" containerName="maas-api" Jul 07 13:30:00.141530 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141531 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2682b17-fa32-40c9-ba9b-55a39aa9ba48" containerName="maas-api" Jul 07 13:30:00.141741 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141546 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099e9658-aedb-427c-a8e9-cae9aad0e6ce" containerName="main" Jul 07 13:30:00.141741 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141551 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="099e9658-aedb-427c-a8e9-cae9aad0e6ce" containerName="main" Jul 07 13:30:00.141741 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141608 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2682b17-fa32-40c9-ba9b-55a39aa9ba48" containerName="maas-api" Jul 07 13:30:00.141741 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141617 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="099e9658-aedb-427c-a8e9-cae9aad0e6ce" containerName="main" Jul 07 13:30:00.141741 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141624 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aa8f76e-be41-4a47-aa12-3c3184387edf" containerName="maas-api" Jul 07 13:30:00.141741 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.141633 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="63bd50f8-780a-4f66-acd1-5c4575da3800" containerName="main" Jul 07 13:30:00.146044 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.146026 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" Jul 07 13:30:00.149157 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.149135 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w6kvh\"" Jul 07 13:30:00.155421 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.155398 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-dhtzz"] Jul 07 13:30:00.183019 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.182985 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cfm6\" (UniqueName: \"kubernetes.io/projected/83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df-kube-api-access-7cfm6\") pod \"maas-api-key-cleanup-29723850-dhtzz\" (UID: \"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df\") " pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" Jul 07 13:30:00.283601 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.283567 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cfm6\" (UniqueName: \"kubernetes.io/projected/83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df-kube-api-access-7cfm6\") pod \"maas-api-key-cleanup-29723850-dhtzz\" (UID: \"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df\") " pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" Jul 07 13:30:00.293424 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.293396 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cfm6\" (UniqueName: \"kubernetes.io/projected/83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df-kube-api-access-7cfm6\") pod \"maas-api-key-cleanup-29723850-dhtzz\" (UID: \"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df\") " pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" Jul 07 13:30:00.457875 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.457779 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" Jul 07 13:30:00.591246 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:00.591222 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-dhtzz"] Jul 07 13:30:00.593540 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:30:00.593512 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83fd8a6d_e3e2_43c9_bfa8_59a71c77a0df.slice/crio-3aae6000ee5141cff9f3ddd78daff3cd5594f4b124d706f6a53be8de5602b9ab WatchSource:0}: Error finding container 3aae6000ee5141cff9f3ddd78daff3cd5594f4b124d706f6a53be8de5602b9ab: Status 404 returned error can't find the container with id 3aae6000ee5141cff9f3ddd78daff3cd5594f4b124d706f6a53be8de5602b9ab Jul 07 13:30:01.200251 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:01.200167 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" event={"ID":"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df","Type":"ContainerStarted","Data":"ec8b53e593891907fd342bd0a10c9c8911132c42160f820c8753e477c0c2286a"} Jul 07 13:30:01.200251 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:01.200218 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" event={"ID":"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df","Type":"ContainerStarted","Data":"3aae6000ee5141cff9f3ddd78daff3cd5594f4b124d706f6a53be8de5602b9ab"} Jul 07 13:30:01.217147 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:01.217008 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" podStartSLOduration=1.216989646 podStartE2EDuration="1.216989646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:30:01.216433189 +0000 UTC m=+2483.201281664" watchObservedRunningTime="2026-07-07 13:30:01.216989646 +0000 UTC m=+2483.201838119" Jul 07 13:30:02.206271 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:02.206233 2568 generic.go:356] "Generic (PLEG): container finished" podID="83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df" containerID="ec8b53e593891907fd342bd0a10c9c8911132c42160f820c8753e477c0c2286a" exitCode=0 Jul 07 13:30:02.206647 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:02.206286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" event={"ID":"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df","Type":"ContainerDied","Data":"ec8b53e593891907fd342bd0a10c9c8911132c42160f820c8753e477c0c2286a"} Jul 07 13:30:03.337503 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:03.337480 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" Jul 07 13:30:03.410489 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:03.410451 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cfm6\" (UniqueName: \"kubernetes.io/projected/83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df-kube-api-access-7cfm6\") pod \"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df\" (UID: \"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df\") " Jul 07 13:30:03.412334 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:03.412303 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df-kube-api-access-7cfm6" (OuterVolumeSpecName: "kube-api-access-7cfm6") pod "83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df" (UID: "83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df"). InnerVolumeSpecName "kube-api-access-7cfm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:30:03.511933 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:03.511898 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cfm6\" (UniqueName: \"kubernetes.io/projected/83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df-kube-api-access-7cfm6\") on node \"ip-10-0-131-17.ec2.internal\" DevicePath \"\"" Jul 07 13:30:04.216409 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:04.216380 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" Jul 07 13:30:04.216580 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:04.216376 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-dhtzz" event={"ID":"83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df","Type":"ContainerDied","Data":"3aae6000ee5141cff9f3ddd78daff3cd5594f4b124d706f6a53be8de5602b9ab"} Jul 07 13:30:04.216580 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:04.216509 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3aae6000ee5141cff9f3ddd78daff3cd5594f4b124d706f6a53be8de5602b9ab" Jul 07 13:30:08.424610 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:08.424582 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723835-7nsbr_b3794214-6812-4402-8aee-359bbf2d5dad/cleanup/0.log" Jul 07 13:30:08.540193 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:08.540168 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723850-dhtzz_83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df/cleanup/0.log" Jul 07 13:30:09.329306 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:09.329275 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c99b98b8b-x6xkq_33fe06f2-92cc-4940-bf64-0f769f2bf3ac/copy-manifests/0.log" Jul 07 13:30:09.336422 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:09.336394 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c99b98b8b-x6xkq_33fe06f2-92cc-4940-bf64-0f769f2bf3ac/manager/0.log" Jul 07 13:30:13.445890 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:13.445853 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-57qwx_8c2c61d0-8953-4228-9446-dcb4b2bb73cd/discovery/0.log" Jul 07 13:30:13.685970 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:13.685942 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-677fdffd75-mzmwn_9eae0218-d422-4ae8-b498-5829c3878c36/kube-auth-proxy/0.log" Jul 07 13:30:22.578918 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:22.578887 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jqxc9_275c0b51-3ae0-4606-8672-f047fa541200/global-pull-secret-syncer/0.log" Jul 07 13:30:22.741604 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:22.741576 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qrwz8_d45af9fd-3eb6-4979-87e3-7e084b78aad5/konnectivity-agent/0.log" Jul 07 13:30:22.814927 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:22.814900 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-17.ec2.internal_e331315f490aea5c3012f6891135f171/haproxy/0.log" Jul 07 13:30:30.217481 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.217413 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d08c2688-4a25-4a4a-87c4-256502309af1/alertmanager/0.log" Jul 07 13:30:30.240613 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.240576 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d08c2688-4a25-4a4a-87c4-256502309af1/config-reloader/0.log" Jul 07 13:30:30.265189 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.265167 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d08c2688-4a25-4a4a-87c4-256502309af1/kube-rbac-proxy-web/0.log" Jul 07 13:30:30.291006 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.290987 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d08c2688-4a25-4a4a-87c4-256502309af1/kube-rbac-proxy/0.log" Jul 07 13:30:30.317061 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.317042 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d08c2688-4a25-4a4a-87c4-256502309af1/kube-rbac-proxy-metric/0.log" Jul 07 13:30:30.341711 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.341692 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d08c2688-4a25-4a4a-87c4-256502309af1/prom-label-proxy/0.log" Jul 07 13:30:30.393496 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.393478 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d08c2688-4a25-4a4a-87c4-256502309af1/init-config-reloader/0.log" Jul 07 13:30:30.824035 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.824006 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vcz6t_9eb4ac60-a8e6-437d-b18b-631e3c984e70/node-exporter/0.log" Jul 07 13:30:30.850988 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.850961 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vcz6t_9eb4ac60-a8e6-437d-b18b-631e3c984e70/kube-rbac-proxy/0.log" Jul 07 13:30:30.902474 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.902454 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vcz6t_9eb4ac60-a8e6-437d-b18b-631e3c984e70/init-textfile/0.log" Jul 07 13:30:30.966256 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:30.966193 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sbxpf_a7790c18-eb2c-4f59-8390-1afbbc8a793d/kube-rbac-proxy-main/0.log" Jul 07 13:30:31.012226 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.012208 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sbxpf_a7790c18-eb2c-4f59-8390-1afbbc8a793d/kube-rbac-proxy-self/0.log" Jul 07 13:30:31.060340 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.060312 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sbxpf_a7790c18-eb2c-4f59-8390-1afbbc8a793d/openshift-state-metrics/0.log" Jul 07 13:30:31.121983 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.121919 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d88-effd-43b0-9d05-f57f838092fe/prometheus/0.log" Jul 07 13:30:31.145781 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.145738 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d88-effd-43b0-9d05-f57f838092fe/config-reloader/0.log" Jul 07 13:30:31.170342 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.170326 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d88-effd-43b0-9d05-f57f838092fe/thanos-sidecar/0.log" Jul 07 13:30:31.195647 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.195625 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d88-effd-43b0-9d05-f57f838092fe/kube-rbac-proxy-web/0.log" Jul 07 13:30:31.218732 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.218713 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d88-effd-43b0-9d05-f57f838092fe/kube-rbac-proxy/0.log" Jul 07 13:30:31.241777 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.241757 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d88-effd-43b0-9d05-f57f838092fe/kube-rbac-proxy-thanos/0.log" Jul 07 13:30:31.263047 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.263028 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d88-effd-43b0-9d05-f57f838092fe/init-config-reloader/0.log" Jul 07 13:30:31.294039 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.294021 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-zhz9b_e0358bae-c731-4629-b7c4-6929f09fc980/prometheus-operator/0.log" Jul 07 13:30:31.318067 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.318046 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-zhz9b_e0358bae-c731-4629-b7c4-6929f09fc980/kube-rbac-proxy/0.log" Jul 07 13:30:31.453462 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.453404 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-677f48795b-vpfkc_38dbfefa-b283-4c6b-8b4e-34c26c3b3897/thanos-query/0.log" Jul 07 13:30:31.478152 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.478134 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-677f48795b-vpfkc_38dbfefa-b283-4c6b-8b4e-34c26c3b3897/kube-rbac-proxy-web/0.log" Jul 07 13:30:31.503598 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.503582 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-677f48795b-vpfkc_38dbfefa-b283-4c6b-8b4e-34c26c3b3897/kube-rbac-proxy/0.log" Jul 07 13:30:31.526806 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.526787 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-677f48795b-vpfkc_38dbfefa-b283-4c6b-8b4e-34c26c3b3897/prom-label-proxy/0.log" Jul 07 13:30:31.563439 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.563422 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-677f48795b-vpfkc_38dbfefa-b283-4c6b-8b4e-34c26c3b3897/kube-rbac-proxy-rules/0.log" Jul 07 13:30:31.585986 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:31.585967 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-677f48795b-vpfkc_38dbfefa-b283-4c6b-8b4e-34c26c3b3897/kube-rbac-proxy-metrics/0.log" Jul 07 13:30:33.057146 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.057082 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm"] Jul 07 13:30:33.057870 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.057449 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df" containerName="cleanup" Jul 07 13:30:33.057870 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.057463 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df" containerName="cleanup" Jul 07 13:30:33.057870 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.057530 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df" containerName="cleanup" Jul 07 13:30:33.060605 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.060590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.063180 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.063157 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kfwnx\"/\"kube-root-ca.crt\"" Jul 07 13:30:33.063291 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.063182 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kfwnx\"/\"openshift-service-ca.crt\"" Jul 07 13:30:33.064127 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.064102 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kfwnx\"/\"default-dockercfg-kcgqp\"" Jul 07 13:30:33.071756 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.071737 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm"] Jul 07 13:30:33.146722 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.146693 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-podres\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.146868 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.146731 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-proc\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.146868 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.146758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjgsr\" (UniqueName: \"kubernetes.io/projected/15de45d4-872a-41fd-93c8-7c0d09a19d8e-kube-api-access-bjgsr\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.146868 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.146799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-sys\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.146868 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.146822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-lib-modules\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247494 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-sys\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247616 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247504 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-lib-modules\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247616 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-podres\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247616 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247589 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-sys\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247715 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247617 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-proc\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247715 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjgsr\" (UniqueName: \"kubernetes.io/projected/15de45d4-872a-41fd-93c8-7c0d09a19d8e-kube-api-access-bjgsr\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247715 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247694 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-proc\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247715 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247706 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-lib-modules\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.247843 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.247708 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15de45d4-872a-41fd-93c8-7c0d09a19d8e-podres\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.259016 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.258990 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjgsr\" (UniqueName: \"kubernetes.io/projected/15de45d4-872a-41fd-93c8-7c0d09a19d8e-kube-api-access-bjgsr\") pod \"perf-node-gather-daemonset-8grwm\" (UID: \"15de45d4-872a-41fd-93c8-7c0d09a19d8e\") " pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.371213 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.371135 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:33.505413 ip-10-0-131-17 kubenswrapper[2568]: W0707 13:30:33.505383 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod15de45d4_872a_41fd_93c8_7c0d09a19d8e.slice/crio-51ee6c9f660c0385f588e94ceeffe30dbf6f1f8adb7a219af5bcf31edb97b673 WatchSource:0}: Error finding container 51ee6c9f660c0385f588e94ceeffe30dbf6f1f8adb7a219af5bcf31edb97b673: Status 404 returned error can't find the container with id 51ee6c9f660c0385f588e94ceeffe30dbf6f1f8adb7a219af5bcf31edb97b673 Jul 07 13:30:33.507057 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:33.507021 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm"] Jul 07 13:30:34.354734 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:34.354696 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" event={"ID":"15de45d4-872a-41fd-93c8-7c0d09a19d8e","Type":"ContainerStarted","Data":"683d2845ad7de2f46cccdd1ed03ac472230da5fd0767713fab4ddf2c0a34398e"} Jul 07 13:30:34.354734 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:34.354745 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" event={"ID":"15de45d4-872a-41fd-93c8-7c0d09a19d8e","Type":"ContainerStarted","Data":"51ee6c9f660c0385f588e94ceeffe30dbf6f1f8adb7a219af5bcf31edb97b673"} Jul 07 13:30:34.355296 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:34.354783 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:34.380158 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:34.380100 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" podStartSLOduration=1.380084304 podStartE2EDuration="1.380084304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:30:34.377949555 +0000 UTC m=+2516.362798024" watchObservedRunningTime="2026-07-07 13:30:34.380084304 +0000 UTC m=+2516.364932827" Jul 07 13:30:35.904023 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:35.903992 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w7t4f_9cb8c914-5ba9-4e15-a054-01d42725ad6d/dns/0.log" Jul 07 13:30:35.931880 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:35.931856 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w7t4f_9cb8c914-5ba9-4e15-a054-01d42725ad6d/kube-rbac-proxy/0.log" Jul 07 13:30:35.961363 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:35.961341 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9n6g2_b2be6b55-d608-438f-9bcd-2d8f9849cb16/dns-node-resolver/0.log" Jul 07 13:30:36.715842 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:36.715816 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-42l45_621128a0-56bf-42dc-9566-2ddd2cf5bc58/node-ca/0.log" Jul 07 13:30:37.965682 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:37.965653 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-57qwx_8c2c61d0-8953-4228-9446-dcb4b2bb73cd/discovery/0.log" Jul 07 13:30:38.034714 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:38.034691 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-677fdffd75-mzmwn_9eae0218-d422-4ae8-b498-5829c3878c36/kube-auth-proxy/0.log" Jul 07 13:30:39.055479 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:39.055404 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5hx9w_dcd5eed6-adfa-4d1a-ae8c-3f2189b774f9/serve-healthcheck-canary/0.log" Jul 07 13:30:39.751904 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:39.751876 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shzbb_badecb7c-59ef-4bc4-a8eb-4817217e8409/kube-rbac-proxy/0.log" Jul 07 13:30:39.779542 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:39.779519 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shzbb_badecb7c-59ef-4bc4-a8eb-4817217e8409/exporter/0.log" Jul 07 13:30:39.804173 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:39.804149 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shzbb_badecb7c-59ef-4bc4-a8eb-4817217e8409/extractor/0.log" Jul 07 13:30:40.369074 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:40.369050 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kfwnx/perf-node-gather-daemonset-8grwm" Jul 07 13:30:42.716082 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:42.716055 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723835-7nsbr_b3794214-6812-4402-8aee-359bbf2d5dad/cleanup/0.log" Jul 07 13:30:42.841300 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:42.841275 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723850-dhtzz_83fd8a6d-e3e2-43c9-bfa8-59a71c77a0df/cleanup/0.log" Jul 07 13:30:43.475446 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:43.475417 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c99b98b8b-x6xkq_33fe06f2-92cc-4940-bf64-0f769f2bf3ac/manager/0.log" Jul 07 13:30:43.505772 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:43.505747 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c99b98b8b-x6xkq_33fe06f2-92cc-4940-bf64-0f769f2bf3ac/copy-manifests/0.log" Jul 07 13:30:43.711093 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:43.711063 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-57qwx_8c2c61d0-8953-4228-9446-dcb4b2bb73cd/discovery/0.log" Jul 07 13:30:43.819322 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:43.819286 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-677fdffd75-mzmwn_9eae0218-d422-4ae8-b498-5829c3878c36/kube-auth-proxy/0.log" Jul 07 13:30:45.111782 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:45.111699 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7c57499d8c-62c99_bc4209d9-4c7b-4d88-9adc-cd9cc4c50bfb/manager/0.log" Jul 07 13:30:45.164786 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:45.164740 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gjppg_f7da946a-f6ea-4e65-b953-98e34fb6198c/openshift-lws-operator/0.log" Jul 07 13:30:45.346236 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:45.346206 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-57qwx_8c2c61d0-8953-4228-9446-dcb4b2bb73cd/discovery/0.log" Jul 07 13:30:45.375083 ip-10-0-131-17 kubenswrapper[2568]: I0707 13:30:45.375015 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-677fdffd75-mzmwn_9eae0218-d422-4ae8-b498-5829c3878c36/kube-auth-proxy/0.log"