Jul 20 15:51:01.709953 ip-10-0-142-55 systemd[1]: Starting Kubernetes Kubelet... Jul 20 15:51:02.067835 ip-10-0-142-55 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 15:51:02.067835 ip-10-0-142-55 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 15:51:02.067835 ip-10-0-142-55 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 15:51:02.067835 ip-10-0-142-55 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 15:51:02.067835 ip-10-0-142-55 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 15:51:02.068717 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.068609 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 15:51:02.073657 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073642 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:51:02.073657 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073657 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073661 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073665 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073667 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073670 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073691 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073695 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073698 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073701 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073704 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073706 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073709 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073712 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073714 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073717 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073720 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073722 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073734 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073738 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:51:02.073741 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073741 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073745 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073748 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073751 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073754 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073758 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073761 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073764 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073768 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073770 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073773 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073776 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073778 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073781 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073783 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073786 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073789 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073791 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073794 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:51:02.074205 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073796 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073798 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073801 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073803 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073805 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073808 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073810 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073813 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073816 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073818 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073821 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073823 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073825 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073828 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073831 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073835 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073838 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073840 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073843 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073845 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:51:02.074656 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073848 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073851 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073853 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073855 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073858 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073860 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073863 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073865 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073867 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073870 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073873 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073875 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073879 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073881 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073883 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073886 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073889 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073891 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073895 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073898 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:51:02.075176 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073901 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073904 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073906 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073908 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073911 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073914 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073918 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073921 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073923 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073926 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073928 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073931 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073933 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073936 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073938 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073941 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073943 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073946 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073949 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:51:02.075651 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.073951 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074656 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074663 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074666 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074669 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074688 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074692 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074694 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074697 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074700 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074703 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074705 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074708 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074711 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074714 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074716 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074719 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074721 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074724 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074727 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:51:02.076154 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074729 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074732 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074735 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074740 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074744 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074747 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074750 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074753 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074756 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074759 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074761 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074764 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074767 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074769 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074772 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074774 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074777 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074779 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074783 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:51:02.076652 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074786 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074789 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074791 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074794 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074797 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074799 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074802 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074804 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074807 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074809 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074812 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074815 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074817 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074820 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074822 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074826 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074830 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074833 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074835 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:51:02.077155 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074838 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074841 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074843 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074846 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074848 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074850 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074853 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074855 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074857 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074860 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074862 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074864 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074867 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074872 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074875 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074877 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074879 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074882 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074884 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:51:02.077605 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074887 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074889 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074892 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074895 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074898 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074900 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074903 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074905 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074908 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074910 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074913 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074915 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074917 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074920 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074922 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074925 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074928 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074931 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074935 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074938 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:51:02.078073 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074940 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074943 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.074945 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075301 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075306 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075309 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075316 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075319 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075322 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075324 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075327 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075330 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075333 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075335 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075338 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075341 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075343 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075346 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075348 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:51:02.078584 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075351 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075353 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075356 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075358 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075361 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075363 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075366 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075368 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075370 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075373 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075377 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075380 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075383 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075385 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075388 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075391 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075393 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075397 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075400 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:51:02.079056 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075404 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075406 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075409 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075412 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075414 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075417 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075420 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075423 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075425 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075428 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075431 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075434 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075437 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075439 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075442 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075445 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075447 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075450 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075452 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075454 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:51:02.079505 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075457 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075459 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075462 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075464 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075468 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075471 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075473 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075476 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075479 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075481 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075484 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075487 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075491 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075494 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075496 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075499 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075501 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075504 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075506 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:51:02.080004 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075508 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075511 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075513 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075515 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075518 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075521 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075523 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075526 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075528 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075531 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075533 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075536 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075538 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075541 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075543 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075546 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075548 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075551 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075553 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075556 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:51:02.080475 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075558 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075561 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075563 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075566 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.075568 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075640 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075647 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075653 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075657 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075661 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075665 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075669 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075689 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075693 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075696 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075699 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075703 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075706 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075709 2559 flags.go:64] FLAG: --cgroup-root="" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075712 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075715 2559 flags.go:64] FLAG: --client-ca-file="" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075718 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075720 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075725 2559 flags.go:64] FLAG: --cluster-domain="" Jul 20 15:51:02.080959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075728 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075730 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075733 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075736 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075740 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075743 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075746 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075749 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075752 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075755 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075758 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075761 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075763 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075768 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075771 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075773 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075776 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075779 2559 flags.go:64] FLAG: --enable-server="true" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075782 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075785 2559 flags.go:64] FLAG: --event-burst="100" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075788 2559 flags.go:64] FLAG: --event-qps="50" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075790 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075793 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075796 2559 flags.go:64] FLAG: --eviction-hard="" Jul 20 15:51:02.081532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075799 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075802 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075805 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075808 2559 flags.go:64] FLAG: --eviction-soft="" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075811 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075814 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075816 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075819 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075822 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075825 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075827 2559 flags.go:64] FLAG: --feature-gates="" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075831 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075834 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075837 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075839 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075842 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075845 2559 flags.go:64] FLAG: --help="false" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075848 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075851 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075854 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075857 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075860 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075863 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075866 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 15:51:02.082117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075868 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075871 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075873 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075876 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075879 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075882 2559 flags.go:64] FLAG: --kube-reserved="" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075884 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075887 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075890 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075893 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075896 2559 flags.go:64] FLAG: --lock-file="" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075899 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075902 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075906 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075911 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075914 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075916 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075919 2559 flags.go:64] FLAG: --logging-format="text" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075922 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075925 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075928 2559 flags.go:64] FLAG: --manifest-url="" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075930 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075934 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075937 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075941 2559 flags.go:64] FLAG: --max-pods="110" Jul 20 15:51:02.082734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075944 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075947 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075949 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075952 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075955 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075968 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075971 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075979 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075982 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075985 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075988 2559 flags.go:64] FLAG: --pod-cidr="" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075991 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075996 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.075998 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076001 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076004 2559 flags.go:64] FLAG: --port="10250" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076007 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076010 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07e1ade6772385306" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076013 2559 flags.go:64] FLAG: --qos-reserved="" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076018 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076021 2559 flags.go:64] FLAG: --register-node="true" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076024 2559 flags.go:64] FLAG: --register-with-taints="" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076027 2559 flags.go:64] FLAG: --registry-burst="10" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076030 2559 flags.go:64] FLAG: --registry-qps="5" Jul 20 15:51:02.083372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076032 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076035 2559 flags.go:64] FLAG: --reserved-memory="" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076038 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076041 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076044 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076047 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076050 2559 flags.go:64] FLAG: --runonce="false" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076052 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076055 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076058 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076061 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076064 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076067 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076070 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076073 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076076 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076078 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076081 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076084 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076086 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076089 2559 flags.go:64] FLAG: --system-cgroups="" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076092 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076099 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076101 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076104 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 15:51:02.083971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076109 2559 flags.go:64] FLAG: --tls-min-version="" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076112 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076116 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076119 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076122 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076124 2559 flags.go:64] FLAG: --v="2" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076128 2559 flags.go:64] FLAG: --version="false" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076132 2559 flags.go:64] FLAG: --vmodule="" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076135 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.076138 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076246 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076250 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076253 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076256 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076259 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076262 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076264 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076267 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076270 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076272 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076274 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076277 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:51:02.084585 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076279 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076282 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076284 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076286 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076289 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076291 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076294 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076297 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076300 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076302 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076304 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076307 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076311 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076314 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076317 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076325 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076328 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076330 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076333 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076336 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:51:02.085127 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076338 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076341 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076343 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076346 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076351 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076354 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076357 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076360 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076363 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076365 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076368 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076371 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076373 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076376 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076379 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076381 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076384 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076386 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076389 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:51:02.085610 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076400 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076404 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076407 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076409 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076412 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076415 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076418 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076421 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076424 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076427 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076431 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076435 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076439 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076443 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076447 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076450 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076454 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076457 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076461 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076465 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:51:02.086120 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076468 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076472 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076475 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076479 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076483 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076487 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076490 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076495 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076500 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076504 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076508 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076511 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076517 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076520 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076524 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076527 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076531 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076537 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076542 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:51:02.086599 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076547 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076552 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076557 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076560 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076564 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076567 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076571 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076575 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.076579 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.077340 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.084015 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.084028 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.084043 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.084052 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084125 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:51:02.087092 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084130 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084134 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084137 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084140 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084143 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084146 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084149 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084152 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084155 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084157 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084160 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084162 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084165 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084167 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084170 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084172 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084175 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084178 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084182 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:51:02.087465 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084191 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084195 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084197 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084200 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084203 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084205 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084208 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084211 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084213 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084216 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084219 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084222 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084224 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084226 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084229 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084231 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084234 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084237 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084239 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084241 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:51:02.087937 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084245 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084249 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084252 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084254 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084257 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084259 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084262 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084264 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084267 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084269 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084271 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084274 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084276 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084284 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084287 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084289 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084292 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084294 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084297 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:51:02.088431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084300 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084304 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084308 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084311 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084314 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084316 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084318 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084321 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084324 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084326 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084329 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084331 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084334 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084336 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084338 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084344 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084347 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084349 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084352 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:51:02.088898 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084354 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084357 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084359 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084361 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084364 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084366 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084369 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084371 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084379 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084382 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084385 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084387 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084389 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084392 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084403 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084407 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084409 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084412 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084414 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084417 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:51:02.089380 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084420 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.084425 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084614 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084620 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084623 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084626 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084628 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084631 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084634 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084637 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084639 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084642 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084646 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084650 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084653 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:51:02.089878 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084656 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084658 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084661 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084663 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084665 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084668 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084693 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084696 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084699 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084707 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084710 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084713 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084716 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084718 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084721 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084724 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084728 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084731 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084733 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084736 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:51:02.090243 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084738 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084741 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084744 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084746 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084748 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084751 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084753 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084755 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084758 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084760 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084762 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084765 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084769 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084772 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084774 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084777 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084780 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084783 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084785 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084794 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:51:02.090735 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084796 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084799 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084802 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084804 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084807 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084810 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084812 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084815 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084817 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084820 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084822 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084824 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084827 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084829 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084832 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084834 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084836 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084839 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084841 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:51:02.091237 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084843 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084846 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084848 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084851 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084853 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084856 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084858 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084860 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084862 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084865 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084868 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084870 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084872 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084881 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084884 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084886 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084889 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084891 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084894 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084896 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:51:02.091725 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084898 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084901 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084903 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084906 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084908 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084911 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:02.084913 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.084918 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.085018 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.087087 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.088446 2559 server.go:1019] "Starting client certificate rotation" Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.088556 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.088591 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 15:51:02.092197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.090806 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 15:51:02.092545 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.092248 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 15:51:02.102716 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.102695 2559 log.go:25] "Validated CRI v1 runtime API" Jul 20 15:51:02.104554 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.104538 2559 log.go:25] "Validated CRI v1 image API" Jul 20 15:51:02.107080 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.107063 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 15:51:02.109197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.109173 2559 fs.go:135] Filesystem UUIDs: map[110431cd-ad73-4d58-a6d3-9b0a7ca172c2:/dev/nvme0n1p3 27a32daf-8ba9-4706-a73a-136a7acee15f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 20 15:51:02.109251 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.109197 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 15:51:02.114925 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.114821 2559 manager.go:216] Machine: {Timestamp:2026-07-20 15:51:02.113533228 +0000 UTC m=+0.292210292 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107347 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2882eb51be4505b28807a27db4e0b9 SystemUUID:ec2882eb-51be-4505-b288-07a27db4e0b9 BootID:c69f27da-8434-4bc2-8d87-185a50d77a1e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:68:2a:58:1f:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:68:2a:58:1f:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:7e:3f:6c:1b:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 15:51:02.114925 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.114921 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 15:51:02.115027 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.114995 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 15:51:02.117514 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.117489 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 15:51:02.117643 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.117515 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 15:51:02.118113 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.118100 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 15:51:02.118150 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.118115 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 15:51:02.118150 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.118122 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 15:51:02.119204 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.119194 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 15:51:02.120413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.120401 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 15:51:02.123455 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.123444 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 20 15:51:02.123593 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.123582 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 15:51:02.123644 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.123611 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 20 15:51:02.123644 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.123621 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 15:51:02.123644 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.123634 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 15:51:02.123644 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.123643 2559 kubelet.go:398] "Adding apiserver pod source" Jul 20 15:51:02.123828 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.123652 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 15:51:02.123828 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.123813 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 15:51:02.124106 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.124093 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 15:51:02.124161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.124115 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 15:51:02.126184 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.126171 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 15:51:02.127945 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.127931 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 15:51:02.127997 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.127963 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 15:51:02.128915 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128899 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 15:51:02.128915 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128916 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128922 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128936 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128945 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128951 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128964 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128969 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128975 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128980 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.128995 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 15:51:02.129014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.129000 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 15:51:02.129644 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.129633 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 15:51:02.129644 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.129645 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 15:51:02.132978 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.132963 2559 server.go:1268] "Started kubelet" Jul 20 15:51:02.133232 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.133199 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 15:51:02.133431 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.133386 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 15:51:02.133491 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.133453 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 15:51:02.134528 ip-10-0-142-55 systemd[1]: Started Kubernetes Kubelet. Jul 20 15:51:02.135223 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.135208 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-55.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 15:51:02.136050 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.136021 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 15:51:02.136050 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.136030 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 15:51:02.136727 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.136710 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 20 15:51:02.137966 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.137953 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 15:51:02.138370 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.138349 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 15:51:02.141606 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.140637 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-55.ec2.internal.18c40a4943dbe324 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-55.ec2.internal,UID:ip-10-0-142-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-55.ec2.internal,},FirstTimestamp:2026-07-20 15:51:02.132933412 +0000 UTC m=+0.311610479,LastTimestamp:2026-07-20 15:51:02.132933412 +0000 UTC m=+0.311610479,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-55.ec2.internal,}" Jul 20 15:51:02.143834 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.143815 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 15:51:02.143922 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.143810 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 15:51:02.143978 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.143933 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.144498 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.144480 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 15:51:02.144636 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.144482 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 15:51:02.144636 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.144634 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 15:51:02.144799 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.144721 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 20 15:51:02.144799 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.144729 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 20 15:51:02.145198 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.145139 2559 factory.go:153] Registering CRI-O factory Jul 20 15:51:02.145262 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.145237 2559 factory.go:223] Registration of the crio container factory successfully Jul 20 15:51:02.145294 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.145286 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 15:51:02.145339 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.145295 2559 factory.go:55] Registering systemd factory Jul 20 15:51:02.145339 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.145303 2559 factory.go:223] Registration of the systemd container factory successfully Jul 20 15:51:02.145339 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.145322 2559 factory.go:103] Registering Raw factory Jul 20 15:51:02.145339 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.145335 2559 manager.go:1196] Started watching for new ooms in manager Jul 20 15:51:02.145645 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.145627 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 15:51:02.145731 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.145638 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 15:51:02.145787 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.145775 2559 manager.go:349] Starting recovery of all containers Jul 20 15:51:02.155628 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.155475 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7jjbm" Jul 20 15:51:02.155746 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.155729 2559 manager.go:354] Recovery completed Jul 20 15:51:02.160056 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.160043 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:51:02.162119 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.162105 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:51:02.162182 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.162136 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:51:02.162182 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.162149 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:51:02.162653 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.162636 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 15:51:02.162653 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.162651 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 15:51:02.162814 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.162660 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:51:02.162814 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.162667 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 20 15:51:02.163298 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.163285 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:51:02.163363 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.163311 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:51:02.163363 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.163321 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:51:02.163923 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.163911 2559 policy_none.go:49] "None policy: Start" Jul 20 15:51:02.163959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.163926 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 15:51:02.163959 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.163936 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 15:51:02.164194 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.164180 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7jjbm" Jul 20 15:51:02.164982 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.164972 2559 policy_none.go:47] "Start" Jul 20 15:51:02.198662 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.198648 2559 manager.go:337] "Starting Device Plugin manager" Jul 20 15:51:02.198767 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.198695 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 15:51:02.198767 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.198708 2559 server.go:87] "Starting device plugin registration server" Jul 20 15:51:02.198936 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.198923 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 15:51:02.198990 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.198937 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 15:51:02.199120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.199105 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 15:51:02.199212 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.199186 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 15:51:02.199212 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.199195 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 15:51:02.199609 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.199589 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 15:51:02.199707 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.199636 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.299033 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.298997 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:51:02.300107 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.299858 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:51:02.300107 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.299886 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:51:02.300107 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.299896 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:51:02.300107 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.299920 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.310083 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.310068 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.310174 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.310089 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-55.ec2.internal\": node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.310215 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.310162 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 15:51:02.311463 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.311444 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 15:51:02.311528 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.311475 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 15:51:02.311528 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.311492 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 15:51:02.311602 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.311528 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 15:51:02.313879 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.313856 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:51:02.347476 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.347431 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.412129 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.412108 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal"] Jul 20 15:51:02.412197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.412174 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:51:02.413455 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.413440 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:51:02.413541 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.413469 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:51:02.413541 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.413482 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:51:02.414718 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.414704 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:51:02.414826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.414810 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.414873 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.414858 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:51:02.415396 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.415365 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:51:02.415396 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.415391 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:51:02.415539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.415401 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:51:02.415539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.415405 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:51:02.415539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.415427 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:51:02.415539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.415447 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:51:02.416710 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.416695 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.416799 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.416724 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:51:02.417398 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.417380 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:51:02.417464 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.417407 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:51:02.417464 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.417422 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:51:02.442392 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.442371 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-55.ec2.internal\" not found" node="ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.445478 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.445462 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3aeda92685e962237fff1ffaa6fe9d8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal\" (UID: \"b3aeda92685e962237fff1ffaa6fe9d8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.445550 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.445495 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3aeda92685e962237fff1ffaa6fe9d8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal\" (UID: \"b3aeda92685e962237fff1ffaa6fe9d8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.445550 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.445523 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/127f11aaeef3ff4f0262a9565d901f2f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-55.ec2.internal\" (UID: \"127f11aaeef3ff4f0262a9565d901f2f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.446686 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.446658 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-55.ec2.internal\" not found" node="ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.447648 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.447632 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.546070 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.546046 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3aeda92685e962237fff1ffaa6fe9d8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal\" (UID: \"b3aeda92685e962237fff1ffaa6fe9d8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.546152 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.546070 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3aeda92685e962237fff1ffaa6fe9d8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal\" (UID: \"b3aeda92685e962237fff1ffaa6fe9d8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.546152 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.546094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/127f11aaeef3ff4f0262a9565d901f2f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-55.ec2.internal\" (UID: \"127f11aaeef3ff4f0262a9565d901f2f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.546152 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.546130 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/127f11aaeef3ff4f0262a9565d901f2f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-55.ec2.internal\" (UID: \"127f11aaeef3ff4f0262a9565d901f2f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.546152 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.546146 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3aeda92685e962237fff1ffaa6fe9d8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal\" (UID: \"b3aeda92685e962237fff1ffaa6fe9d8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.546276 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.546156 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3aeda92685e962237fff1ffaa6fe9d8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal\" (UID: \"b3aeda92685e962237fff1ffaa6fe9d8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.548181 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.548164 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.648959 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.648903 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.745177 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.745154 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.749080 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.749010 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.750127 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:02.750113 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" Jul 20 15:51:02.849104 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.849074 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:02.949654 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:02.949588 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:03.050006 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:03.049980 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:03.073995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.073976 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:51:03.088189 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.088168 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 15:51:03.088280 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.088262 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 15:51:03.088330 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.088295 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 15:51:03.150844 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:03.150820 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:03.167687 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.167641 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 15:46:02 +0000 UTC" deadline="2028-05-01 06:50:49.86299869 +0000 UTC" Jul 20 15:51:03.167687 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.167666 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15614h59m46.695335194s" Jul 20 15:51:03.251272 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:03.251247 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:03.302419 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.302392 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:51:03.352163 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:03.352136 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:03.352597 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:03.352561 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod127f11aaeef3ff4f0262a9565d901f2f.slice/crio-e6731705da713ca8cd1b90b162e738d35cfd7671a97930850f5bd82a2cdf2732 WatchSource:0}: Error finding container e6731705da713ca8cd1b90b162e738d35cfd7671a97930850f5bd82a2cdf2732: Status 404 returned error can't find the container with id e6731705da713ca8cd1b90b162e738d35cfd7671a97930850f5bd82a2cdf2732 Jul 20 15:51:03.356704 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.356673 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 15:51:03.362088 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:03.362063 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3aeda92685e962237fff1ffaa6fe9d8.slice/crio-171c05be96fcf4d7ea1aff4cc1164e0d108f0b85b635cad0956836b80f8efb0c WatchSource:0}: Error finding container 171c05be96fcf4d7ea1aff4cc1164e0d108f0b85b635cad0956836b80f8efb0c: Status 404 returned error can't find the container with id 171c05be96fcf4d7ea1aff4cc1164e0d108f0b85b635cad0956836b80f8efb0c Jul 20 15:51:03.452364 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:03.452328 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:03.553482 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:03.553404 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:03.653862 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:03.653832 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-55.ec2.internal\" not found" Jul 20 15:51:03.688205 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.688177 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:51:03.747614 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.747582 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" Jul 20 15:51:03.766976 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.766953 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 15:51:03.768010 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.767997 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" Jul 20 15:51:03.784334 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:03.784310 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 15:51:04.035272 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.035182 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:51:04.125102 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.124833 2559 apiserver.go:52] "Watching apiserver" Jul 20 15:51:04.131976 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.131952 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 15:51:04.133971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.133942 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-zk9nm","openshift-image-registry/node-ca-8mc5h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal","openshift-multus/multus-9mzwb","openshift-network-diagnostics/network-check-target-gxvpj","kube-system/konnectivity-agent-vqp9q","openshift-dns/node-resolver-qjhtn","openshift-multus/multus-additional-cni-plugins-vk5sb","openshift-multus/network-metrics-daemon-l9db4","openshift-network-operator/iptables-alerter-9xlqf","openshift-ovn-kubernetes/ovnkube-node-6686p","kube-system/global-pull-secret-syncer-kjkb7","kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr"] Jul 20 15:51:04.136909 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.136889 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.140787 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.140756 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.141314 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.141288 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 15:51:04.141717 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.141682 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8x5sk\"" Jul 20 15:51:04.141847 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.141828 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 15:51:04.143949 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.143926 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 15:51:04.144078 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.144060 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 15:51:04.144272 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.144254 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lwk2q\"" Jul 20 15:51:04.144376 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.144357 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 15:51:04.144867 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.144847 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 15:51:04.145477 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.145455 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 15:51:04.145839 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.145819 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 15:51:04.145940 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.145917 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.148216 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.148195 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 15:51:04.148825 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.148801 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9c7cd\"" Jul 20 15:51:04.149300 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.149279 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 15:51:04.150262 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.150239 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 15:51:04.150353 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.150300 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 15:51:04.152108 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.152084 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.154702 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.154676 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 15:51:04.154971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.154952 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 15:51:04.155090 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155073 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ctmzv\"" Jul 20 15:51:04.155521 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155479 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvpz2\" (UniqueName: \"kubernetes.io/projected/ca77ff59-3d9a-4552-bb24-9b515e3adca5-kube-api-access-qvpz2\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.155521 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155516 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-run-netns\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155546 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-run-ovn-kubernetes\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155572 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-sys-fs\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155596 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-cni-bin\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155614 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-etc-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155629 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-cnibin\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155648 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-kubelet\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155664 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-hostroot\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155678 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-conf-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.155719 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155692 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-multus-certs\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155706 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-etc-kubernetes\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155754 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwbj9\" (UniqueName: \"kubernetes.io/projected/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-kube-api-access-bwbj9\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155778 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-device-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155800 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-daemon-config\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-kubelet\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155861 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovn-node-metrics-cert\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155888 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-socket-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155914 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-etc-selinux\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155957 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-k8s-cni-cncf-io\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.155982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-systemd-units\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156017 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-slash\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156041 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156074 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-cni-bin\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156100 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvp5m\" (UniqueName: \"kubernetes.io/projected/09634e74-5e78-4cba-903b-4377e0f5464e-kube-api-access-pvp5m\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156124 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-cni-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.156208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156147 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovnkube-config\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156171 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovnkube-script-lib\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156199 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqf5k\" (UniqueName: \"kubernetes.io/projected/b14f91e0-7393-4a56-a2a5-3146e3d57975-kube-api-access-wqf5k\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156227 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-os-release\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-socket-dir-parent\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156292 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-var-lib-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156332 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-ovn\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156368 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-node-log\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156394 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-log-socket\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156425 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156451 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156497 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-registration-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156537 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-system-cni-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca77ff59-3d9a-4552-bb24-9b515e3adca5-hosts-file\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156680 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca77ff59-3d9a-4552-bb24-9b515e3adca5-tmp-dir\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156740 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-systemd\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156790 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:04.157076 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156789 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-cni-binary-copy\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.157927 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.156898 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:04.157927 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156920 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-netns\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.157927 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156949 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-cni-multus\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.157927 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.156971 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-cni-netd\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.157927 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.157007 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-env-overrides\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.158326 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.158271 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 15:51:04.159587 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.159562 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:04.160304 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.160283 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 15:51:04.160728 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.160684 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 15:51:04.162292 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.162257 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:04.162389 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.162327 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:04.162632 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.162608 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-75fhk\"" Jul 20 15:51:04.162914 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.162894 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 15:51:04.163120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.163105 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 15:51:04.164793 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.164728 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.167201 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.167183 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.170020 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.170001 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 15:51:04.170248 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.170231 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 15:51:04.170466 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.170446 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 15:51:04.170590 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.170575 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-95ztr\"" Jul 20 15:51:04.170702 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.170681 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 15:51:04.170937 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.170788 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nbf4b\"" Jul 20 15:51:04.170937 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.170790 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 15:51:04.171125 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.170987 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.173862 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.173842 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.177272 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.177246 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s9vx6\"" Jul 20 15:51:04.177523 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.177506 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 15:51:04.177741 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.177703 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 15:51:04.177948 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.177930 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 15:51:04.178111 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.178093 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.178203 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.178159 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:04.178203 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.178197 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hf8bp\"" Jul 20 15:51:04.178593 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.178579 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 15:51:04.178828 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.178785 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 15:51:04.189187 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.189170 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v6s4n" Jul 20 15:51:04.198896 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.198874 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v6s4n" Jul 20 15:51:04.246672 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.246649 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 15:51:04.257471 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257442 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.257596 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257491 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovnkube-script-lib\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.257596 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-os-release\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.257813 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257705 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-os-release\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.257813 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257761 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-cnibin\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.258067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.258067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257875 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-ovn\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.258067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257902 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.258067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257929 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-registration-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.258067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.257999 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-ovn\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.258067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258002 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-system-cni-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.258067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258058 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258058 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-netns\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258101 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-netns\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-multus-certs\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258136 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-multus-certs\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258148 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovnkube-script-lib\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258149 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258188 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysconfig\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258203 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-system-cni-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258216 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca77ff59-3d9a-4552-bb24-9b515e3adca5-tmp-dir\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258238 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-registration-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258242 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-systemd\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-cni-binary-copy\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258294 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-daemon-config\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258335 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e504f4ca-eda2-4a94-b105-b608e047dffa-host-slash\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258352 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-systemd\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258358 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysctl-conf\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258411 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-cni-netd\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.258539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258438 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-env-overrides\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258463 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-etc-kubernetes\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258490 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-system-cni-dir\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258518 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg56c\" (UniqueName: \"kubernetes.io/projected/56e694e2-a3a4-43ac-9460-738beb3e48d5-kube-api-access-bg56c\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258546 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnhhb\" (UniqueName: \"kubernetes.io/projected/112add68-aa85-4112-b49a-95a66a50498c-kube-api-access-wnhhb\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258573 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-cnibin\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258597 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-kubelet\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258621 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-hostroot\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-serviceca\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258672 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-modprobe-d\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258696 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-systemd\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258742 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-device-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258769 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258794 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-sys\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258821 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovn-node-metrics-cert\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258846 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfb70808-2749-4ee4-bc2d-df7d8a15f643-konnectivity-ca\") pod \"konnectivity-agent-vqp9q\" (UID: \"cfb70808-2749-4ee4-bc2d-df7d8a15f643\") " pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258872 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-os-release\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.259425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258895 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-cni-binary-copy\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258906 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysctl-d\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.258929 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/112add68-aa85-4112-b49a-95a66a50498c-tmp\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259253 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca77ff59-3d9a-4552-bb24-9b515e3adca5-tmp-dir\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259323 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-cni-netd\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259401 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-daemon-config\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259474 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvp5m\" (UniqueName: \"kubernetes.io/projected/09634e74-5e78-4cba-903b-4377e0f5464e-kube-api-access-pvp5m\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259502 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-cni-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259529 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3d9ab51-3b57-4749-9de3-4724174739f4-dbus\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259558 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovnkube-config\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqf5k\" (UniqueName: \"kubernetes.io/projected/b14f91e0-7393-4a56-a2a5-3146e3d57975-kube-api-access-wqf5k\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259613 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-socket-dir-parent\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259641 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdsw5\" (UniqueName: \"kubernetes.io/projected/e504f4ca-eda2-4a94-b105-b608e047dffa-kube-api-access-cdsw5\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259669 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-lib-modules\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259676 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-env-overrides\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259691 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-var-lib-kubelet\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.260284 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259723 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-etc-kubernetes\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259773 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-run-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259778 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-cnibin\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259829 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-kubelet\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259878 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-hostroot\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.259959 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-device-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260076 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-socket-dir-parent\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260136 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-host\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260167 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260208 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-var-lib-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260280 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-node-log\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-log-socket\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260304 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260334 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260361 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-cni-multus\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260387 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-conf-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260412 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcqbk\" (UniqueName: \"kubernetes.io/projected/9e8bc050-0d54-4bf9-977f-3496292834bd-kube-api-access-qcqbk\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260461 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca77ff59-3d9a-4552-bb24-9b515e3adca5-hosts-file\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.261181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260485 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/112add68-aa85-4112-b49a-95a66a50498c-etc-tuned\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260513 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvpz2\" (UniqueName: \"kubernetes.io/projected/ca77ff59-3d9a-4552-bb24-9b515e3adca5-kube-api-access-qvpz2\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-run-netns\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260565 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-run-ovn-kubernetes\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-sys-fs\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260614 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-cni-bin\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260639 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbj9\" (UniqueName: \"kubernetes.io/projected/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-kube-api-access-bwbj9\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-etc-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260688 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-host\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260764 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260797 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl29h\" (UniqueName: \"kubernetes.io/projected/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-kube-api-access-fl29h\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260848 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-kubernetes\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260871 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-run\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-kubelet\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260925 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-socket-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260950 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-etc-selinux\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.262161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.260979 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-k8s-cni-cncf-io\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261005 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e504f4ca-eda2-4a94-b105-b608e047dffa-iptables-alerter-script\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261033 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfb70808-2749-4ee4-bc2d-df7d8a15f643-agent-certs\") pod \"konnectivity-agent-vqp9q\" (UID: \"cfb70808-2749-4ee4-bc2d-df7d8a15f643\") " pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261063 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3d9ab51-3b57-4749-9de3-4724174739f4-kubelet-config\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261089 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-systemd-units\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-slash\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-cni-bin\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261241 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-cni-bin\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261251 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-sys-fs\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261294 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-var-lib-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261296 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-cni-bin\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261515 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-etc-openvswitch\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261585 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-kubelet\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261696 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-socket-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261819 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-etc-selinux\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261818 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovnkube-config\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261863 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-node-log\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.263157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261877 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-run-k8s-cni-cncf-io\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261936 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-systemd-units\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261938 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-slash\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.261997 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca77ff59-3d9a-4552-bb24-9b515e3adca5-hosts-file\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.262044 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-cni-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.262059 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-log-socket\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.262140 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09634e74-5e78-4cba-903b-4377e0f5464e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.262180 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-host-var-lib-cni-multus\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.262216 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-multus-conf-dir\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.262252 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-run-netns\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.262269 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b14f91e0-7393-4a56-a2a5-3146e3d57975-host-run-ovn-kubernetes\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.265120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.264966 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b14f91e0-7393-4a56-a2a5-3146e3d57975-ovn-node-metrics-cert\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.270247 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.270228 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvp5m\" (UniqueName: \"kubernetes.io/projected/09634e74-5e78-4cba-903b-4377e0f5464e-kube-api-access-pvp5m\") pod \"aws-ebs-csi-driver-node-rs6cr\" (UID: \"09634e74-5e78-4cba-903b-4377e0f5464e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.276007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.275958 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqf5k\" (UniqueName: \"kubernetes.io/projected/b14f91e0-7393-4a56-a2a5-3146e3d57975-kube-api-access-wqf5k\") pod \"ovnkube-node-6686p\" (UID: \"b14f91e0-7393-4a56-a2a5-3146e3d57975\") " pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.277989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.277964 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvpz2\" (UniqueName: \"kubernetes.io/projected/ca77ff59-3d9a-4552-bb24-9b515e3adca5-kube-api-access-qvpz2\") pod \"node-resolver-qjhtn\" (UID: \"ca77ff59-3d9a-4552-bb24-9b515e3adca5\") " pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.278612 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.278570 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwbj9\" (UniqueName: \"kubernetes.io/projected/41f2c090-5b68-4e4e-a09f-7c693bc3bab4-kube-api-access-bwbj9\") pod \"multus-9mzwb\" (UID: \"41f2c090-5b68-4e4e-a09f-7c693bc3bab4\") " pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.316573 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.316492 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" event={"ID":"127f11aaeef3ff4f0262a9565d901f2f","Type":"ContainerStarted","Data":"e6731705da713ca8cd1b90b162e738d35cfd7671a97930850f5bd82a2cdf2732"} Jul 20 15:51:04.317955 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.317928 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" event={"ID":"b3aeda92685e962237fff1ffaa6fe9d8","Type":"ContainerStarted","Data":"171c05be96fcf4d7ea1aff4cc1164e0d108f0b85b635cad0956836b80f8efb0c"} Jul 20 15:51:04.361967 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.361942 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-cnibin\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.362090 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.361971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.362090 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362053 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-cnibin\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.362214 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:04.362214 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362132 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysconfig\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362214 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362163 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e504f4ca-eda2-4a94-b105-b608e047dffa-host-slash\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.362214 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362190 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysctl-conf\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362218 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-system-cni-dir\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.362413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg56c\" (UniqueName: \"kubernetes.io/projected/56e694e2-a3a4-43ac-9460-738beb3e48d5-kube-api-access-bg56c\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.362413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362273 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnhhb\" (UniqueName: \"kubernetes.io/projected/112add68-aa85-4112-b49a-95a66a50498c-kube-api-access-wnhhb\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362291 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e504f4ca-eda2-4a94-b105-b608e047dffa-host-slash\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.362413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362303 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-serviceca\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.362413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362339 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-modprobe-d\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362366 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-systemd\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362395 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362420 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-sys\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfb70808-2749-4ee4-bc2d-df7d8a15f643-konnectivity-ca\") pod \"konnectivity-agent-vqp9q\" (UID: \"cfb70808-2749-4ee4-bc2d-df7d8a15f643\") " pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-os-release\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysctl-d\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362522 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/112add68-aa85-4112-b49a-95a66a50498c-tmp\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362551 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3d9ab51-3b57-4749-9de3-4724174739f4-dbus\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362581 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdsw5\" (UniqueName: \"kubernetes.io/projected/e504f4ca-eda2-4a94-b105-b608e047dffa-kube-api-access-cdsw5\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362606 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-lib-modules\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362630 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-var-lib-kubelet\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362653 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-host\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362678 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362672 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362716 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcqbk\" (UniqueName: \"kubernetes.io/projected/9e8bc050-0d54-4bf9-977f-3496292834bd-kube-api-access-qcqbk\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:04.362826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362811 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-serviceca\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362870 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/112add68-aa85-4112-b49a-95a66a50498c-etc-tuned\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.362913 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362949 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-host\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.362994 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret podName:c3d9ab51-3b57-4749-9de3-4724174739f4 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:04.862957501 +0000 UTC m=+3.041634575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret") pod "global-pull-secret-syncer-kjkb7" (UID: "c3d9ab51-3b57-4749-9de3-4724174739f4") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.362916 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-host\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363108 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl29h\" (UniqueName: \"kubernetes.io/projected/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-kube-api-access-fl29h\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363164 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-kubernetes\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363201 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-system-cni-dir\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363218 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-run\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.363238 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363249 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e504f4ca-eda2-4a94-b105-b608e047dffa-iptables-alerter-script\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363252 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-var-lib-kubelet\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363322 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-kubernetes\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363355 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-lib-modules\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.363482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363420 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-os-release\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.363523 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs podName:9e8bc050-0d54-4bf9-977f-3496292834bd nodeName:}" failed. No retries permitted until 2026-07-20 15:51:04.863468007 +0000 UTC m=+3.042145075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs") pod "network-metrics-daemon-l9db4" (UID: "9e8bc050-0d54-4bf9-977f-3496292834bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363645 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-modprobe-d\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363693 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-systemd\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363821 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e504f4ca-eda2-4a94-b105-b608e047dffa-iptables-alerter-script\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363937 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-sys\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363943 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysctl-d\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364005 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364054 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysconfig\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.363166 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-etc-sysctl-conf\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364307 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-run\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.364338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364347 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/112add68-aa85-4112-b49a-95a66a50498c-host\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.364989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364455 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfb70808-2749-4ee4-bc2d-df7d8a15f643-konnectivity-ca\") pod \"konnectivity-agent-vqp9q\" (UID: \"cfb70808-2749-4ee4-bc2d-df7d8a15f643\") " pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:04.364989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364542 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfb70808-2749-4ee4-bc2d-df7d8a15f643-agent-certs\") pod \"konnectivity-agent-vqp9q\" (UID: \"cfb70808-2749-4ee4-bc2d-df7d8a15f643\") " pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:04.364989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364565 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3d9ab51-3b57-4749-9de3-4724174739f4-kubelet-config\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.364989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.364989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364759 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3d9ab51-3b57-4749-9de3-4724174739f4-dbus\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.364989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364826 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56e694e2-a3a4-43ac-9460-738beb3e48d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.364989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364881 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3d9ab51-3b57-4749-9de3-4724174739f4-kubelet-config\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.365222 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.364991 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56e694e2-a3a4-43ac-9460-738beb3e48d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.366999 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.366974 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/112add68-aa85-4112-b49a-95a66a50498c-etc-tuned\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.367087 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.367003 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/112add68-aa85-4112-b49a-95a66a50498c-tmp\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.367459 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.367440 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfb70808-2749-4ee4-bc2d-df7d8a15f643-agent-certs\") pod \"konnectivity-agent-vqp9q\" (UID: \"cfb70808-2749-4ee4-bc2d-df7d8a15f643\") " pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:04.377745 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.377717 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl29h\" (UniqueName: \"kubernetes.io/projected/37fdc62c-c7ed-46c0-a651-a2cc2a420cb0-kube-api-access-fl29h\") pod \"node-ca-8mc5h\" (UID: \"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0\") " pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.378437 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.378388 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnhhb\" (UniqueName: \"kubernetes.io/projected/112add68-aa85-4112-b49a-95a66a50498c-kube-api-access-wnhhb\") pod \"tuned-zk9nm\" (UID: \"112add68-aa85-4112-b49a-95a66a50498c\") " pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.378864 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.378846 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:51:04.378964 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.378869 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:51:04.378964 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.378884 2559 projected.go:196] Error preparing data for projected volume kube-api-access-8v2ts for pod openshift-network-diagnostics/network-check-target-gxvpj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:04.378964 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.378956 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts podName:ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a nodeName:}" failed. No retries permitted until 2026-07-20 15:51:04.878938857 +0000 UTC m=+3.057615932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8v2ts" (UniqueName: "kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts") pod "network-check-target-gxvpj" (UID: "ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:04.381342 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.380955 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcqbk\" (UniqueName: \"kubernetes.io/projected/9e8bc050-0d54-4bf9-977f-3496292834bd-kube-api-access-qcqbk\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:04.381342 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.381146 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg56c\" (UniqueName: \"kubernetes.io/projected/56e694e2-a3a4-43ac-9460-738beb3e48d5-kube-api-access-bg56c\") pod \"multus-additional-cni-plugins-vk5sb\" (UID: \"56e694e2-a3a4-43ac-9460-738beb3e48d5\") " pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.381342 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.381307 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdsw5\" (UniqueName: \"kubernetes.io/projected/e504f4ca-eda2-4a94-b105-b608e047dffa-kube-api-access-cdsw5\") pod \"iptables-alerter-9xlqf\" (UID: \"e504f4ca-eda2-4a94-b105-b608e047dffa\") " pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.453592 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.453562 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qjhtn" Jul 20 15:51:04.464689 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.464660 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:04.476003 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.475977 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" Jul 20 15:51:04.483885 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.483857 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9mzwb" Jul 20 15:51:04.492609 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.492590 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:04.500612 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.500590 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" Jul 20 15:51:04.510425 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.510408 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8mc5h" Jul 20 15:51:04.529159 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.529137 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9xlqf" Jul 20 15:51:04.535984 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.535965 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" Jul 20 15:51:04.868865 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.868827 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:04.869039 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.868892 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:04.869039 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.869008 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:04.869039 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.869008 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:04.869188 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.869079 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret podName:c3d9ab51-3b57-4749-9de3-4724174739f4 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:05.869060076 +0000 UTC m=+4.047737129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret") pod "global-pull-secret-syncer-kjkb7" (UID: "c3d9ab51-3b57-4749-9de3-4724174739f4") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:04.869188 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.869098 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs podName:9e8bc050-0d54-4bf9-977f-3496292834bd nodeName:}" failed. No retries permitted until 2026-07-20 15:51:05.869088106 +0000 UTC m=+4.047765158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs") pod "network-metrics-daemon-l9db4" (UID: "9e8bc050-0d54-4bf9-977f-3496292834bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:04.969910 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:04.969876 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:04.970099 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.970006 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:51:04.970099 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.970020 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:51:04.970099 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.970032 2559 projected.go:196] Error preparing data for projected volume kube-api-access-8v2ts for pod openshift-network-diagnostics/network-check-target-gxvpj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:04.970099 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:04.970092 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts podName:ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a nodeName:}" failed. No retries permitted until 2026-07-20 15:51:05.970077225 +0000 UTC m=+4.148754278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8v2ts" (UniqueName: "kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts") pod "network-check-target-gxvpj" (UID: "ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:05.202917 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:05.202810 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 15:46:04 +0000 UTC" deadline="2027-12-23 22:39:56.711506393 +0000 UTC" Jul 20 15:51:05.202917 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:05.202872 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12510h48m51.508639428s" Jul 20 15:51:05.312620 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:05.312582 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:05.312829 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:05.312699 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:05.312829 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.312707 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:05.312829 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.312794 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:05.501555 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:05.501521 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112add68_aa85_4112_b49a_95a66a50498c.slice/crio-ac7b96761a232838fe45d08358a2c0478d983a4d1310f56c2e47f59fbdd9c44a WatchSource:0}: Error finding container ac7b96761a232838fe45d08358a2c0478d983a4d1310f56c2e47f59fbdd9c44a: Status 404 returned error can't find the container with id ac7b96761a232838fe45d08358a2c0478d983a4d1310f56c2e47f59fbdd9c44a Jul 20 15:51:05.674258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:05.674061 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:51:05.877880 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:05.877844 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:05.878030 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:05.877910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:05.878030 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.877980 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:05.878030 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.878008 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:05.878162 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.878040 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs podName:9e8bc050-0d54-4bf9-977f-3496292834bd nodeName:}" failed. No retries permitted until 2026-07-20 15:51:07.878023321 +0000 UTC m=+6.056700373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs") pod "network-metrics-daemon-l9db4" (UID: "9e8bc050-0d54-4bf9-977f-3496292834bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:05.878162 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.878068 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret podName:c3d9ab51-3b57-4749-9de3-4724174739f4 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:07.878047965 +0000 UTC m=+6.056725028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret") pod "global-pull-secret-syncer-kjkb7" (UID: "c3d9ab51-3b57-4749-9de3-4724174739f4") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:05.979643 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:05.979615 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:05.979797 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.979772 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:51:05.979797 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.979786 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:51:05.979797 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.979795 2559 projected.go:196] Error preparing data for projected volume kube-api-access-8v2ts for pod openshift-network-diagnostics/network-check-target-gxvpj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:05.979904 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:05.979842 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts podName:ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a nodeName:}" failed. No retries permitted until 2026-07-20 15:51:07.979827372 +0000 UTC m=+6.158504424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8v2ts" (UniqueName: "kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts") pod "network-check-target-gxvpj" (UID: "ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:06.203249 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.203137 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 15:46:04 +0000 UTC" deadline="2028-04-22 13:35:35.462685866 +0000 UTC" Jul 20 15:51:06.203249 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.203173 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15405h44m29.259516926s" Jul 20 15:51:06.315112 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.314483 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:06.315112 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:06.314612 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:06.336469 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.336434 2559 generic.go:356] "Generic (PLEG): container finished" podID="b3aeda92685e962237fff1ffaa6fe9d8" containerID="7ffc84599f34a701f59882ae0cba7212e0cc96b0ec1cd3b28f984d705d9b24a6" exitCode=0 Jul 20 15:51:06.336632 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.336540 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" event={"ID":"b3aeda92685e962237fff1ffaa6fe9d8","Type":"ContainerDied","Data":"7ffc84599f34a701f59882ae0cba7212e0cc96b0ec1cd3b28f984d705d9b24a6"} Jul 20 15:51:06.346429 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.346397 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" event={"ID":"127f11aaeef3ff4f0262a9565d901f2f","Type":"ContainerStarted","Data":"833982d5e55d62597617c2d7b12c838d8cbc4ee27ee2fdcb941dd2814fe0b3a3"} Jul 20 15:51:06.374910 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.374788 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9xlqf" event={"ID":"e504f4ca-eda2-4a94-b105-b608e047dffa","Type":"ContainerStarted","Data":"c48d3b408bf7135de6a9cbc6fa69bd7b8d05846c9b300318f7d681f5d292278f"} Jul 20 15:51:06.390462 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.390380 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" event={"ID":"09634e74-5e78-4cba-903b-4377e0f5464e","Type":"ContainerStarted","Data":"38f4b8936e1ba2108c77c03953b86413e8cf471c444aad0b92129daacc717438"} Jul 20 15:51:06.426703 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.426672 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vqp9q" event={"ID":"cfb70808-2749-4ee4-bc2d-df7d8a15f643","Type":"ContainerStarted","Data":"2d107ea2ecd697a6218c1e3f57c48d4bbdc68c1247f906b232ca0a1467a10b65"} Jul 20 15:51:06.436593 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.436482 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qjhtn" event={"ID":"ca77ff59-3d9a-4552-bb24-9b515e3adca5","Type":"ContainerStarted","Data":"f2fa3b7d983d15fdcf60c2884c11b30b3c352d2ef78ea3f1d4b4262a8dfcd7d0"} Jul 20 15:51:06.451934 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.451904 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" event={"ID":"56e694e2-a3a4-43ac-9460-738beb3e48d5","Type":"ContainerStarted","Data":"9845482325a1fb1f3e26c46f4ee2049faae06770f6c92c46a2e0de14c0dfe875"} Jul 20 15:51:06.478214 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.478148 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"2d94c39a8c494267d9dc3c2c197289ebbe8162bb007771769a50411876e5bc16"} Jul 20 15:51:06.484348 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.484323 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8mc5h" event={"ID":"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0","Type":"ContainerStarted","Data":"e9c57f6802a22896440f4ecabb49e4f076105e4aeeb412a7cb6ffb0b81a12a0d"} Jul 20 15:51:06.491231 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.491206 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9mzwb" event={"ID":"41f2c090-5b68-4e4e-a09f-7c693bc3bab4","Type":"ContainerStarted","Data":"4b6e94f389de11297aab8ac4c1f39f1f37348255885e44664cf45803e7f52513"} Jul 20 15:51:06.499322 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:06.499267 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" event={"ID":"112add68-aa85-4112-b49a-95a66a50498c","Type":"ContainerStarted","Data":"ac7b96761a232838fe45d08358a2c0478d983a4d1310f56c2e47f59fbdd9c44a"} Jul 20 15:51:07.312469 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:07.312416 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:07.313220 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.312556 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:07.313220 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:07.312609 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:07.313220 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.312770 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:07.521863 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:07.521814 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" event={"ID":"b3aeda92685e962237fff1ffaa6fe9d8","Type":"ContainerStarted","Data":"1ecaffa012d3b3d052b8da7f885a74c658e39fe105c813c706936695f72318e6"} Jul 20 15:51:07.541850 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:07.540769 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-55.ec2.internal" podStartSLOduration=3.540747285 podStartE2EDuration="3.540747285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:07.540588139 +0000 UTC m=+5.719265216" watchObservedRunningTime="2026-07-20 15:51:07.540747285 +0000 UTC m=+5.719424363" Jul 20 15:51:07.541850 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:07.541607 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-55.ec2.internal" podStartSLOduration=3.541594467 podStartE2EDuration="3.541594467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:06.37277048 +0000 UTC m=+4.551447560" watchObservedRunningTime="2026-07-20 15:51:07.541594467 +0000 UTC m=+5.720271547" Jul 20 15:51:07.897078 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:07.897034 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:07.897300 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:07.897119 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:07.897300 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.897283 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:07.897416 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.897349 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs podName:9e8bc050-0d54-4bf9-977f-3496292834bd nodeName:}" failed. No retries permitted until 2026-07-20 15:51:11.897328089 +0000 UTC m=+10.076005157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs") pod "network-metrics-daemon-l9db4" (UID: "9e8bc050-0d54-4bf9-977f-3496292834bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:07.897794 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.897775 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:07.897909 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.897833 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret podName:c3d9ab51-3b57-4749-9de3-4724174739f4 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:11.897817252 +0000 UTC m=+10.076494308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret") pod "global-pull-secret-syncer-kjkb7" (UID: "c3d9ab51-3b57-4749-9de3-4724174739f4") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:07.998396 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:07.998277 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:07.998574 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.998500 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:51:07.998574 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.998531 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:51:07.998574 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.998547 2559 projected.go:196] Error preparing data for projected volume kube-api-access-8v2ts for pod openshift-network-diagnostics/network-check-target-gxvpj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:07.998823 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:07.998609 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts podName:ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a nodeName:}" failed. No retries permitted until 2026-07-20 15:51:11.998588983 +0000 UTC m=+10.177266062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8v2ts" (UniqueName: "kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts") pod "network-check-target-gxvpj" (UID: "ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:08.315670 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:08.315069 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:08.315670 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:08.315247 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:09.312570 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:09.312535 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:09.312795 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:09.312693 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:09.313119 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:09.313097 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:09.313270 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:09.313226 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:10.312308 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:10.312161 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:10.312308 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:10.312306 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:11.311737 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:11.311703 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:11.311907 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:11.311834 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:11.311907 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:11.311870 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:11.312044 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:11.312003 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:11.932534 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:11.932502 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:11.933048 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:11.932567 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:11.933048 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:11.932741 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:11.933048 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:11.932804 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs podName:9e8bc050-0d54-4bf9-977f-3496292834bd nodeName:}" failed. No retries permitted until 2026-07-20 15:51:19.93278401 +0000 UTC m=+18.111461063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs") pod "network-metrics-daemon-l9db4" (UID: "9e8bc050-0d54-4bf9-977f-3496292834bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:11.933270 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:11.933250 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:11.933348 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:11.933313 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret podName:c3d9ab51-3b57-4749-9de3-4724174739f4 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:19.933296874 +0000 UTC m=+18.111973927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret") pod "global-pull-secret-syncer-kjkb7" (UID: "c3d9ab51-3b57-4749-9de3-4724174739f4") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:12.033314 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:12.033270 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:12.033502 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:12.033477 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:51:12.033594 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:12.033508 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:51:12.033594 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:12.033523 2559 projected.go:196] Error preparing data for projected volume kube-api-access-8v2ts for pod openshift-network-diagnostics/network-check-target-gxvpj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:12.033594 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:12.033582 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts podName:ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a nodeName:}" failed. No retries permitted until 2026-07-20 15:51:20.033562361 +0000 UTC m=+18.212239416 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8v2ts" (UniqueName: "kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts") pod "network-check-target-gxvpj" (UID: "ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:12.313176 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:12.313083 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:12.313368 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:12.313222 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:13.312086 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:13.312037 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:13.312570 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:13.312200 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:13.312639 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:13.312584 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:13.312701 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:13.312687 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:14.312062 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:14.312031 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:14.313122 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:14.312449 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:15.312503 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:15.312466 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:15.312663 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:15.312605 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:15.313067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:15.313015 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:15.313255 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:15.313119 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:16.313505 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:16.313477 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:16.313890 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:16.313565 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:17.312012 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:17.311960 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:17.312089 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:17.312059 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:17.312137 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:17.312100 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:17.312189 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:17.312152 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:18.312549 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:18.312500 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:18.312988 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:18.312668 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:19.311919 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:19.311887 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:19.312132 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:19.312006 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:19.312132 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:19.312065 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:19.312250 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:19.312182 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:19.999914 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:19.999880 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:20.000364 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:19.999937 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:20.000364 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.000038 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:20.000364 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.000055 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:20.000364 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.000107 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret podName:c3d9ab51-3b57-4749-9de3-4724174739f4 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:36.000087647 +0000 UTC m=+34.178764770 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret") pod "global-pull-secret-syncer-kjkb7" (UID: "c3d9ab51-3b57-4749-9de3-4724174739f4") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:20.000364 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.000127 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs podName:9e8bc050-0d54-4bf9-977f-3496292834bd nodeName:}" failed. No retries permitted until 2026-07-20 15:51:36.000117455 +0000 UTC m=+34.178794519 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs") pod "network-metrics-daemon-l9db4" (UID: "9e8bc050-0d54-4bf9-977f-3496292834bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:20.101005 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:20.100967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:20.101183 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.101159 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:51:20.101237 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.101191 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:51:20.101237 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.101202 2559 projected.go:196] Error preparing data for projected volume kube-api-access-8v2ts for pod openshift-network-diagnostics/network-check-target-gxvpj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:20.101322 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.101264 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts podName:ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a nodeName:}" failed. No retries permitted until 2026-07-20 15:51:36.101235826 +0000 UTC m=+34.279912877 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8v2ts" (UniqueName: "kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts") pod "network-check-target-gxvpj" (UID: "ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:20.315120 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:20.315093 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:20.315285 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:20.315195 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:21.311775 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:21.311744 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:21.312195 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:21.311745 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:21.312195 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:21.311872 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:21.312195 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:21.312011 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:22.316546 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:22.316520 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:22.316943 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:22.316630 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:23.312361 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:23.312333 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:23.312519 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:23.312370 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:23.312519 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:23.312443 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:23.312643 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:23.312595 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:24.312704 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:24.312656 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:24.313254 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:24.312801 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:25.311947 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.311920 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:25.312075 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.311949 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:25.312141 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:25.312068 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:25.312215 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:25.312191 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:25.566247 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.565968 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" event={"ID":"09634e74-5e78-4cba-903b-4377e0f5464e","Type":"ContainerStarted","Data":"d7fd963992b5cdc35d8bcb2174ed721bc778788bc06adf2717dd9791afdade34"} Jul 20 15:51:25.567489 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.567461 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vqp9q" event={"ID":"cfb70808-2749-4ee4-bc2d-df7d8a15f643","Type":"ContainerStarted","Data":"9a242f86fe01f5fadb8bbd303626b7b5b67a0e860d4d72b045c5d09feb3a3646"} Jul 20 15:51:25.569046 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.569023 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qjhtn" event={"ID":"ca77ff59-3d9a-4552-bb24-9b515e3adca5","Type":"ContainerStarted","Data":"143e9c5249496986eb1f762aa055243a51dd0b649532b0c5105f4e5fe6baa45e"} Jul 20 15:51:25.570422 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.570397 2559 generic.go:356] "Generic (PLEG): container finished" podID="56e694e2-a3a4-43ac-9460-738beb3e48d5" containerID="1111e74d53579139bb4daf85c00a86f69d9141bfce6afc542aaadd5be8df9711" exitCode=0 Jul 20 15:51:25.570521 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.570463 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" event={"ID":"56e694e2-a3a4-43ac-9460-738beb3e48d5","Type":"ContainerDied","Data":"1111e74d53579139bb4daf85c00a86f69d9141bfce6afc542aaadd5be8df9711"} Jul 20 15:51:25.572815 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.572793 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"21673d09afb48b9d36e55c0f5ebc30fa0b255c706ad68089d09894d5058beab2"} Jul 20 15:51:25.572909 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.572851 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"c928418fc4948687d59c76c206a0000873c6d70b8578a493f1469b0e6bae5322"} Jul 20 15:51:25.572909 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.572898 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"898b280c08532153d0794d5a414cf787f6a7ad01bca36bd2992463ff8d928b11"} Jul 20 15:51:25.574349 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.574324 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8mc5h" event={"ID":"37fdc62c-c7ed-46c0-a651-a2cc2a420cb0","Type":"ContainerStarted","Data":"8924806e1a2cea570d04201a629008aeab7eca19d186478e68a8e5b6c935f59a"} Jul 20 15:51:25.575834 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.575817 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9mzwb" event={"ID":"41f2c090-5b68-4e4e-a09f-7c693bc3bab4","Type":"ContainerStarted","Data":"eab7de5e8b8156eee23c51195d11b44e941c9ca9671e7a5d59942d84a2cb6914"} Jul 20 15:51:25.577281 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.577263 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" event={"ID":"112add68-aa85-4112-b49a-95a66a50498c","Type":"ContainerStarted","Data":"15a03a1947680eba4e7af75380bdf6c038799bfe4cf8efeefc8b92629134f220"} Jul 20 15:51:25.584865 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.584830 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vqp9q" podStartSLOduration=7.128520298 podStartE2EDuration="22.584817946s" totalImagesPullingTime="15.456297648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:25.584400517 +0000 UTC m=+23.763077593" watchObservedRunningTime="2026-07-20 15:51:25.584817946 +0000 UTC m=+23.763495018" Jul 20 15:51:25.618701 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.618647 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qjhtn" podStartSLOduration=2.944997186 podStartE2EDuration="22.618633873s" totalImagesPullingTime="19.673636687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:25.597509957 +0000 UTC m=+23.776187029" watchObservedRunningTime="2026-07-20 15:51:25.618633873 +0000 UTC m=+23.797310946" Jul 20 15:51:25.631786 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.631750 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8mc5h" podStartSLOduration=2.958702421 podStartE2EDuration="22.631736077s" totalImagesPullingTime="19.673033656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:25.631335853 +0000 UTC m=+23.810012926" watchObservedRunningTime="2026-07-20 15:51:25.631736077 +0000 UTC m=+23.810413152" Jul 20 15:51:25.647728 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:25.647691 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9mzwb" podStartSLOduration=2.924210564 podStartE2EDuration="22.647660349s" totalImagesPullingTime="19.723449785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:25.647652446 +0000 UTC m=+23.826329519" watchObservedRunningTime="2026-07-20 15:51:25.647660349 +0000 UTC m=+23.826337404" Jul 20 15:51:26.312825 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:26.312795 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:26.312981 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:26.312901 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:26.582410 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:26.582371 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"1b74307b55cf2754b3591a433eb6770000e6f0b851917c5da43431bda7b1fe02"} Jul 20 15:51:26.582775 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:26.582441 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"2f3f593f1ad495840f021c4e1303cca1f25b22fe4dd8983f6d99d709cdb318ea"} Jul 20 15:51:26.582775 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:26.582478 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"e23f80c282415cfc43b952d5ded3cb7a6903830537957ff9d092a7b025db6884"} Jul 20 15:51:26.727950 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:26.727931 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 15:51:27.211171 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.211071 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T15:51:26.727946793Z","UUID":"7d5af271-65fa-4769-8ca5-d1501e17ec65","Handler":null,"Name":"","Endpoint":""} Jul 20 15:51:27.212694 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.212658 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 15:51:27.212830 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.212716 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 15:51:27.312351 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.312325 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:27.312484 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:27.312440 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:27.312552 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.312501 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:27.312646 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:27.312623 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:27.586455 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.586419 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9xlqf" event={"ID":"e504f4ca-eda2-4a94-b105-b608e047dffa","Type":"ContainerStarted","Data":"b76027ee701d02343ef4a2cbcab13630043c538028080d91c86b1dd1c69e67c9"} Jul 20 15:51:27.588733 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.588704 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" event={"ID":"09634e74-5e78-4cba-903b-4377e0f5464e","Type":"ContainerStarted","Data":"625668de44a5af56f7a4822c1d3b3961c576dcd008b36413f604cdec1768f88e"} Jul 20 15:51:27.600743 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.600699 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zk9nm" podStartSLOduration=4.898565379 podStartE2EDuration="24.60066011s" totalImagesPullingTime="19.702094731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:25.66414213 +0000 UTC m=+23.842819207" watchObservedRunningTime="2026-07-20 15:51:27.60066011 +0000 UTC m=+25.779337184" Jul 20 15:51:27.601194 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:27.601158 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9xlqf" podStartSLOduration=4.93107941 podStartE2EDuration="24.601151952s" totalImagesPullingTime="19.670072542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:27.60048339 +0000 UTC m=+25.779160463" watchObservedRunningTime="2026-07-20 15:51:27.601151952 +0000 UTC m=+25.779829025" Jul 20 15:51:28.312125 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:28.312095 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:28.312282 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:28.312220 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:28.593711 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:28.593616 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" event={"ID":"09634e74-5e78-4cba-903b-4377e0f5464e","Type":"ContainerStarted","Data":"cd930879256415eaf0fae15d4ae2ce1fc04bd37cbe0cab2181bdede7023ea85f"} Jul 20 15:51:28.597660 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:28.597628 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"0c1316a51f80aaae8bf9867baf3d32fd9951305e3d07b07a778bd639fb4c5fbf"} Jul 20 15:51:28.613111 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:28.613074 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rs6cr" podStartSLOduration=3.544562687 podStartE2EDuration="25.613059463s" totalImagesPullingTime="22.068496776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:28.611915093 +0000 UTC m=+26.790592169" watchObservedRunningTime="2026-07-20 15:51:28.613059463 +0000 UTC m=+26.791736538" Jul 20 15:51:29.312125 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:29.311961 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:29.312276 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:29.312001 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:29.312276 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:29.312187 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:29.312276 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:29.312259 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:29.497017 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:29.496987 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:29.497892 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:29.497871 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:30.311776 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:30.311741 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:30.312301 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:30.311874 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:31.312136 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.311978 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:31.312639 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.312047 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:31.312639 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:31.312211 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:31.312639 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:31.312333 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:31.606886 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.606837 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" event={"ID":"b14f91e0-7393-4a56-a2a5-3146e3d57975","Type":"ContainerStarted","Data":"e8f7a4e48cbc86becd048c4e752016b3513fb54b5c0c2994c6460fbda23bb7db"} Jul 20 15:51:31.607485 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.607457 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:31.607612 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.607568 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:31.607703 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.607663 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:31.612210 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.612182 2559 generic.go:356] "Generic (PLEG): container finished" podID="56e694e2-a3a4-43ac-9460-738beb3e48d5" containerID="f7b0e9fb15c8dcbc642b8c64cb8b8a61848b0b0194d40d646aaeeb6a6efed9f9" exitCode=0 Jul 20 15:51:31.612332 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.612219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" event={"ID":"56e694e2-a3a4-43ac-9460-738beb3e48d5","Type":"ContainerDied","Data":"f7b0e9fb15c8dcbc642b8c64cb8b8a61848b0b0194d40d646aaeeb6a6efed9f9"} Jul 20 15:51:31.626172 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.626149 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:31.626433 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.626420 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:51:31.637264 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:31.637225 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" podStartSLOduration=8.892891784 podStartE2EDuration="28.637213784s" totalImagesPullingTime="19.744322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:31.636514799 +0000 UTC m=+29.815191874" watchObservedRunningTime="2026-07-20 15:51:31.637213784 +0000 UTC m=+29.815890851" Jul 20 15:51:32.313532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:32.313490 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:32.313907 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:32.313608 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:33.312124 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:33.312096 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:33.312279 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:33.312191 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:33.312279 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:33.312256 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:33.312394 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:33.312366 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:33.618441 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:33.618368 2559 generic.go:356] "Generic (PLEG): container finished" podID="56e694e2-a3a4-43ac-9460-738beb3e48d5" containerID="ab45606cd0ebecdb5c915dc3563f5abea597a6450867c52800fa2668bdf357b4" exitCode=0 Jul 20 15:51:33.618825 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:33.618454 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" event={"ID":"56e694e2-a3a4-43ac-9460-738beb3e48d5","Type":"ContainerDied","Data":"ab45606cd0ebecdb5c915dc3563f5abea597a6450867c52800fa2668bdf357b4"} Jul 20 15:51:33.940296 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:33.940240 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:33.940420 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:33.940357 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 20 15:51:33.940737 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:33.940713 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vqp9q" Jul 20 15:51:34.311929 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:34.311900 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:34.312103 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:34.312000 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:35.312031 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:35.312005 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:35.312366 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:35.312118 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:35.312366 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:35.312169 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:35.312366 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:35.312244 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:35.625524 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:35.625459 2559 generic.go:356] "Generic (PLEG): container finished" podID="56e694e2-a3a4-43ac-9460-738beb3e48d5" containerID="f48bfed9472cc6cb7741bb25b2b03ad8f7efe7e2d734d96f59d3dc42926914ad" exitCode=0 Jul 20 15:51:35.625524 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:35.625512 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" event={"ID":"56e694e2-a3a4-43ac-9460-738beb3e48d5","Type":"ContainerDied","Data":"f48bfed9472cc6cb7741bb25b2b03ad8f7efe7e2d734d96f59d3dc42926914ad"} Jul 20 15:51:36.022973 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:36.022887 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:36.022973 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:36.022930 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:36.023193 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.023026 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:36.023193 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.023026 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:36.023193 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.023077 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs podName:9e8bc050-0d54-4bf9-977f-3496292834bd nodeName:}" failed. No retries permitted until 2026-07-20 15:52:08.023062415 +0000 UTC m=+66.201739466 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs") pod "network-metrics-daemon-l9db4" (UID: "9e8bc050-0d54-4bf9-977f-3496292834bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:51:36.023193 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.023095 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret podName:c3d9ab51-3b57-4749-9de3-4724174739f4 nodeName:}" failed. No retries permitted until 2026-07-20 15:52:08.023085955 +0000 UTC m=+66.201763029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret") pod "global-pull-secret-syncer-kjkb7" (UID: "c3d9ab51-3b57-4749-9de3-4724174739f4") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:51:36.123993 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:36.123960 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:36.124136 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.124074 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:51:36.124136 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.124090 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:51:36.124136 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.124098 2559 projected.go:196] Error preparing data for projected volume kube-api-access-8v2ts for pod openshift-network-diagnostics/network-check-target-gxvpj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:36.124271 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.124150 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts podName:ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a nodeName:}" failed. No retries permitted until 2026-07-20 15:52:08.124131687 +0000 UTC m=+66.302808738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8v2ts" (UniqueName: "kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts") pod "network-check-target-gxvpj" (UID: "ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:51:36.312691 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:36.312643 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:36.313074 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:36.312800 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:37.030020 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:37.027425 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9db4"] Jul 20 15:51:37.030020 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:37.027559 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:37.030020 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:37.027695 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:37.030708 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:37.030612 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gxvpj"] Jul 20 15:51:37.030708 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:37.030665 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kjkb7"] Jul 20 15:51:37.030708 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:37.030762 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:37.031012 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:37.030858 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:37.031012 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:37.030912 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:37.031012 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:37.030983 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:38.311875 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:38.311699 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:38.312258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:38.311700 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:38.312258 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:38.311950 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:38.312258 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:38.312040 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:39.312165 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:39.312125 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:39.312622 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:39.312259 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kjkb7" podUID="c3d9ab51-3b57-4749-9de3-4724174739f4" Jul 20 15:51:40.312082 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:40.312051 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:40.312241 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:40.312169 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxvpj" podUID="ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a" Jul 20 15:51:40.312241 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:40.312217 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:40.312524 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:40.312315 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9db4" podUID="9e8bc050-0d54-4bf9-977f-3496292834bd" Jul 20 15:51:41.169428 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.169406 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-55.ec2.internal" event="NodeReady" Jul 20 15:51:41.169579 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.169520 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 15:51:41.216868 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.216838 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ktzcc"] Jul 20 15:51:41.227563 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.227544 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-frsgc"] Jul 20 15:51:41.227695 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.227662 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ktzcc" Jul 20 15:51:41.230258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.230236 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z5q65\"" Jul 20 15:51:41.230642 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.230623 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 15:51:41.230729 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.230703 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 15:51:41.230789 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.230760 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 15:51:41.238234 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.238215 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ktzcc"] Jul 20 15:51:41.238302 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.238246 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-frsgc"] Jul 20 15:51:41.238343 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.238322 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.240258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.240241 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 15:51:41.240444 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.240429 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 15:51:41.240539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.240474 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 15:51:41.240539 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.240479 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d2lnj\"" Jul 20 15:51:41.240662 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.240550 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 15:51:41.263592 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.263572 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e0170e8-2d46-490e-aecf-ccccc6eb7b73-cert\") pod \"ingress-canary-ktzcc\" (UID: \"1e0170e8-2d46-490e-aecf-ccccc6eb7b73\") " pod="openshift-ingress-canary/ingress-canary-ktzcc" Jul 20 15:51:41.263715 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.263600 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfrln\" (UniqueName: \"kubernetes.io/projected/f32c9452-5c16-43fd-8612-4c19214966b9-kube-api-access-dfrln\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.263715 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.263620 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32c9452-5c16-43fd-8612-4c19214966b9-crio-socket\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.263715 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.263637 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c9452-5c16-43fd-8612-4c19214966b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.263715 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.263657 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qr7z\" (UniqueName: \"kubernetes.io/projected/1e0170e8-2d46-490e-aecf-ccccc6eb7b73-kube-api-access-4qr7z\") pod \"ingress-canary-ktzcc\" (UID: \"1e0170e8-2d46-490e-aecf-ccccc6eb7b73\") " pod="openshift-ingress-canary/ingress-canary-ktzcc" Jul 20 15:51:41.263715 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.263710 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c9452-5c16-43fd-8612-4c19214966b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.263935 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.263811 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32c9452-5c16-43fd-8612-4c19214966b9-data-volume\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.312545 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.312522 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:51:41.314246 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.314226 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 15:51:41.316566 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.316548 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9vmxs"] Jul 20 15:51:41.324345 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.324330 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.326090 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.326073 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5lhc8\"" Jul 20 15:51:41.326182 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.326163 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 15:51:41.326241 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.326214 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 15:51:41.328356 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.328337 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9vmxs"] Jul 20 15:51:41.364752 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364722 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31cf61c8-7ab9-4a0b-9995-be5987ffde36-metrics-tls\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.364868 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364790 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e0170e8-2d46-490e-aecf-ccccc6eb7b73-cert\") pod \"ingress-canary-ktzcc\" (UID: \"1e0170e8-2d46-490e-aecf-ccccc6eb7b73\") " pod="openshift-ingress-canary/ingress-canary-ktzcc" Jul 20 15:51:41.364868 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364826 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfrln\" (UniqueName: \"kubernetes.io/projected/f32c9452-5c16-43fd-8612-4c19214966b9-kube-api-access-dfrln\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.364868 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32c9452-5c16-43fd-8612-4c19214966b9-crio-socket\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.365023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364883 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c9452-5c16-43fd-8612-4c19214966b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.365023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364909 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31cf61c8-7ab9-4a0b-9995-be5987ffde36-config-volume\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.365023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qr7z\" (UniqueName: \"kubernetes.io/projected/1e0170e8-2d46-490e-aecf-ccccc6eb7b73-kube-api-access-4qr7z\") pod \"ingress-canary-ktzcc\" (UID: \"1e0170e8-2d46-490e-aecf-ccccc6eb7b73\") " pod="openshift-ingress-canary/ingress-canary-ktzcc" Jul 20 15:51:41.365023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364970 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c9452-5c16-43fd-8612-4c19214966b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.365023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.364993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31cf61c8-7ab9-4a0b-9995-be5987ffde36-tmp-dir\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.365267 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.365045 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hbrn\" (UniqueName: \"kubernetes.io/projected/31cf61c8-7ab9-4a0b-9995-be5987ffde36-kube-api-access-6hbrn\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.365267 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.365075 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32c9452-5c16-43fd-8612-4c19214966b9-data-volume\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.365267 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.365082 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32c9452-5c16-43fd-8612-4c19214966b9-crio-socket\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.365496 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.365469 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c9452-5c16-43fd-8612-4c19214966b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.365634 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.365621 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32c9452-5c16-43fd-8612-4c19214966b9-data-volume\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.368767 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.368747 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c9452-5c16-43fd-8612-4c19214966b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.368885 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.368868 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e0170e8-2d46-490e-aecf-ccccc6eb7b73-cert\") pod \"ingress-canary-ktzcc\" (UID: \"1e0170e8-2d46-490e-aecf-ccccc6eb7b73\") " pod="openshift-ingress-canary/ingress-canary-ktzcc" Jul 20 15:51:41.372725 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.372660 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfrln\" (UniqueName: \"kubernetes.io/projected/f32c9452-5c16-43fd-8612-4c19214966b9-kube-api-access-dfrln\") pod \"insights-runtime-extractor-frsgc\" (UID: \"f32c9452-5c16-43fd-8612-4c19214966b9\") " pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.372823 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.372798 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qr7z\" (UniqueName: \"kubernetes.io/projected/1e0170e8-2d46-490e-aecf-ccccc6eb7b73-kube-api-access-4qr7z\") pod \"ingress-canary-ktzcc\" (UID: \"1e0170e8-2d46-490e-aecf-ccccc6eb7b73\") " pod="openshift-ingress-canary/ingress-canary-ktzcc" Jul 20 15:51:41.465638 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.465586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31cf61c8-7ab9-4a0b-9995-be5987ffde36-tmp-dir\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.465750 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.465635 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hbrn\" (UniqueName: \"kubernetes.io/projected/31cf61c8-7ab9-4a0b-9995-be5987ffde36-kube-api-access-6hbrn\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.465750 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.465670 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31cf61c8-7ab9-4a0b-9995-be5987ffde36-metrics-tls\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.465865 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.465764 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31cf61c8-7ab9-4a0b-9995-be5987ffde36-config-volume\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.465909 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.465878 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31cf61c8-7ab9-4a0b-9995-be5987ffde36-tmp-dir\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.466232 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.466212 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31cf61c8-7ab9-4a0b-9995-be5987ffde36-config-volume\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.467636 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.467620 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31cf61c8-7ab9-4a0b-9995-be5987ffde36-metrics-tls\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.472612 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.472589 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hbrn\" (UniqueName: \"kubernetes.io/projected/31cf61c8-7ab9-4a0b-9995-be5987ffde36-kube-api-access-6hbrn\") pod \"dns-default-9vmxs\" (UID: \"31cf61c8-7ab9-4a0b-9995-be5987ffde36\") " pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.537573 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.537550 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ktzcc" Jul 20 15:51:41.549049 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.549034 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-frsgc" Jul 20 15:51:41.632981 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.632803 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:41.648850 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.648737 2559 generic.go:356] "Generic (PLEG): container finished" podID="56e694e2-a3a4-43ac-9460-738beb3e48d5" containerID="5b657569d4d1f9ad54dff04f3379c8297379ac62dad23381c50182a9ae372055" exitCode=0 Jul 20 15:51:41.648850 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.648799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" event={"ID":"56e694e2-a3a4-43ac-9460-738beb3e48d5","Type":"ContainerDied","Data":"5b657569d4d1f9ad54dff04f3379c8297379ac62dad23381c50182a9ae372055"} Jul 20 15:51:41.726572 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.726498 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ktzcc"] Jul 20 15:51:41.728168 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.728072 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-frsgc"] Jul 20 15:51:41.733571 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:41.733532 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0170e8_2d46_490e_aecf_ccccc6eb7b73.slice/crio-d9e3667e3fd77d1dda5c25dc34a59da0a8eb28c639aae4e0988d513a5ef8111b WatchSource:0}: Error finding container d9e3667e3fd77d1dda5c25dc34a59da0a8eb28c639aae4e0988d513a5ef8111b: Status 404 returned error can't find the container with id d9e3667e3fd77d1dda5c25dc34a59da0a8eb28c639aae4e0988d513a5ef8111b Jul 20 15:51:41.740545 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:41.740516 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32c9452_5c16_43fd_8612_4c19214966b9.slice/crio-c42258d20a53814bd945cba348481721734d3702438260227613e650b450559f WatchSource:0}: Error finding container c42258d20a53814bd945cba348481721734d3702438260227613e650b450559f: Status 404 returned error can't find the container with id c42258d20a53814bd945cba348481721734d3702438260227613e650b450559f Jul 20 15:51:41.814493 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:41.814422 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9vmxs"] Jul 20 15:51:41.819630 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:41.819603 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31cf61c8_7ab9_4a0b_9995_be5987ffde36.slice/crio-17f87b1f1d29b0a5195fac32bccf3de56e3a47071f696e92aef3f2b74232361f WatchSource:0}: Error finding container 17f87b1f1d29b0a5195fac32bccf3de56e3a47071f696e92aef3f2b74232361f: Status 404 returned error can't find the container with id 17f87b1f1d29b0a5195fac32bccf3de56e3a47071f696e92aef3f2b74232361f Jul 20 15:51:42.313849 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.311926 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:51:42.313849 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.313666 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:51:42.320222 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.320197 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6j7cm\"" Jul 20 15:51:42.321565 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.321540 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 15:51:42.321840 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.321801 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7vt6p\"" Jul 20 15:51:42.322549 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.322380 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 15:51:42.322549 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.322426 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 15:51:42.657137 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.657062 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ktzcc" event={"ID":"1e0170e8-2d46-490e-aecf-ccccc6eb7b73","Type":"ContainerStarted","Data":"d9e3667e3fd77d1dda5c25dc34a59da0a8eb28c639aae4e0988d513a5ef8111b"} Jul 20 15:51:42.666491 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.666454 2559 generic.go:356] "Generic (PLEG): container finished" podID="56e694e2-a3a4-43ac-9460-738beb3e48d5" containerID="d86058d1e39e1878e8dedf9bdcff64afc3a5f07007fdd7ef4aec4ac5ba8d5a87" exitCode=0 Jul 20 15:51:42.666624 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.666546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" event={"ID":"56e694e2-a3a4-43ac-9460-738beb3e48d5","Type":"ContainerDied","Data":"d86058d1e39e1878e8dedf9bdcff64afc3a5f07007fdd7ef4aec4ac5ba8d5a87"} Jul 20 15:51:42.673842 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.673777 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9vmxs" event={"ID":"31cf61c8-7ab9-4a0b-9995-be5987ffde36","Type":"ContainerStarted","Data":"17f87b1f1d29b0a5195fac32bccf3de56e3a47071f696e92aef3f2b74232361f"} Jul 20 15:51:42.675347 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:42.675288 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-frsgc" event={"ID":"f32c9452-5c16-43fd-8612-4c19214966b9","Type":"ContainerStarted","Data":"c42258d20a53814bd945cba348481721734d3702438260227613e650b450559f"} Jul 20 15:51:43.685442 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:43.684731 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" event={"ID":"56e694e2-a3a4-43ac-9460-738beb3e48d5","Type":"ContainerStarted","Data":"55afb6dcf48c1f5c3a9b0dc927a145a56ae0be065b98f3ba29638153481c5ef5"} Jul 20 15:51:43.707268 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:43.707217 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vk5sb" podStartSLOduration=8.475644697 podStartE2EDuration="40.707200732s" totalImagesPullingTime="32.231556035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:43.706086121 +0000 UTC m=+41.884763193" watchObservedRunningTime="2026-07-20 15:51:43.707200732 +0000 UTC m=+41.885877805" Jul 20 15:51:44.952915 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:44.952868 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-fnnpd"] Jul 20 15:51:44.966804 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:44.966778 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-fnnpd"] Jul 20 15:51:44.966936 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:44.966897 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-fnnpd" Jul 20 15:51:44.968836 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:44.968815 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 15:51:44.968971 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:44.968915 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 15:51:44.969131 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:44.969099 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wwjff\"" Jul 20 15:51:45.001907 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:45.001821 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgswx\" (UniqueName: \"kubernetes.io/projected/cbf42dd3-8ef3-48b3-86f7-5a66f14814dc-kube-api-access-rgswx\") pod \"downloads-5bc67dcd46-fnnpd\" (UID: \"cbf42dd3-8ef3-48b3-86f7-5a66f14814dc\") " pod="openshift-console/downloads-5bc67dcd46-fnnpd" Jul 20 15:51:45.102823 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:45.102796 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgswx\" (UniqueName: \"kubernetes.io/projected/cbf42dd3-8ef3-48b3-86f7-5a66f14814dc-kube-api-access-rgswx\") pod \"downloads-5bc67dcd46-fnnpd\" (UID: \"cbf42dd3-8ef3-48b3-86f7-5a66f14814dc\") " pod="openshift-console/downloads-5bc67dcd46-fnnpd" Jul 20 15:51:45.116636 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:45.116617 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgswx\" (UniqueName: \"kubernetes.io/projected/cbf42dd3-8ef3-48b3-86f7-5a66f14814dc-kube-api-access-rgswx\") pod \"downloads-5bc67dcd46-fnnpd\" (UID: \"cbf42dd3-8ef3-48b3-86f7-5a66f14814dc\") " pod="openshift-console/downloads-5bc67dcd46-fnnpd" Jul 20 15:51:45.278088 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:45.278032 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-fnnpd" Jul 20 15:51:46.114597 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.114563 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr"] Jul 20 15:51:46.243108 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.243065 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr"] Jul 20 15:51:46.243266 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.243218 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.245756 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.245543 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 15:51:46.245756 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.245561 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-g9v8j\"" Jul 20 15:51:46.245756 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.245543 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 20 15:51:46.246071 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.246051 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 15:51:46.246281 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.246265 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 20 15:51:46.246497 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.246479 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 15:51:46.311300 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.311270 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3f580e1-fd44-41dd-8eb0-124be7af2870-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.311443 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.311340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3f580e1-fd44-41dd-8eb0-124be7af2870-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.311443 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.311374 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvczm\" (UniqueName: \"kubernetes.io/projected/a3f580e1-fd44-41dd-8eb0-124be7af2870-kube-api-access-cvczm\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.311443 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.311406 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3f580e1-fd44-41dd-8eb0-124be7af2870-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.411995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.411917 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3f580e1-fd44-41dd-8eb0-124be7af2870-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.411995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.411978 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3f580e1-fd44-41dd-8eb0-124be7af2870-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.412218 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.412016 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvczm\" (UniqueName: \"kubernetes.io/projected/a3f580e1-fd44-41dd-8eb0-124be7af2870-kube-api-access-cvczm\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.412218 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.412054 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3f580e1-fd44-41dd-8eb0-124be7af2870-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.412822 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.412747 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3f580e1-fd44-41dd-8eb0-124be7af2870-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.414796 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.414655 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3f580e1-fd44-41dd-8eb0-124be7af2870-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.414796 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.414735 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3f580e1-fd44-41dd-8eb0-124be7af2870-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.421447 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.421426 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvczm\" (UniqueName: \"kubernetes.io/projected/a3f580e1-fd44-41dd-8eb0-124be7af2870-kube-api-access-cvczm\") pod \"prometheus-operator-5945b5cfd-fhzbr\" (UID: \"a3f580e1-fd44-41dd-8eb0-124be7af2870\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:46.554738 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:46.554712 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" Jul 20 15:51:48.777664 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:48.777634 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr"] Jul 20 15:51:48.783161 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:48.783133 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f580e1_fd44_41dd_8eb0_124be7af2870.slice/crio-1f664bea08cb960937e08acbc7c9db6e95a7e134f916967acb82c70ea7decfcf WatchSource:0}: Error finding container 1f664bea08cb960937e08acbc7c9db6e95a7e134f916967acb82c70ea7decfcf: Status 404 returned error can't find the container with id 1f664bea08cb960937e08acbc7c9db6e95a7e134f916967acb82c70ea7decfcf Jul 20 15:51:48.798660 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:48.798578 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-fnnpd"] Jul 20 15:51:48.803138 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:48.803110 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf42dd3_8ef3_48b3_86f7_5a66f14814dc.slice/crio-eb9a80aab4efdbe416f93f88d6687b50b6c6723d8f6a1a133a3cbaa6877ac431 WatchSource:0}: Error finding container eb9a80aab4efdbe416f93f88d6687b50b6c6723d8f6a1a133a3cbaa6877ac431: Status 404 returned error can't find the container with id eb9a80aab4efdbe416f93f88d6687b50b6c6723d8f6a1a133a3cbaa6877ac431 Jul 20 15:51:49.711168 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:49.711126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ktzcc" event={"ID":"1e0170e8-2d46-490e-aecf-ccccc6eb7b73","Type":"ContainerStarted","Data":"2ef7dcbd413c08cb52dc5802f62d8e5c9006f193e605f2891a3e2d01d42d739c"} Jul 20 15:51:49.713196 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:49.713141 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-fnnpd" event={"ID":"cbf42dd3-8ef3-48b3-86f7-5a66f14814dc","Type":"ContainerStarted","Data":"eb9a80aab4efdbe416f93f88d6687b50b6c6723d8f6a1a133a3cbaa6877ac431"} Jul 20 15:51:49.717035 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:49.717011 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9vmxs" event={"ID":"31cf61c8-7ab9-4a0b-9995-be5987ffde36","Type":"ContainerStarted","Data":"f57c1f16547fddc307e3e3cbde44445892d864be868c4cfee06a3ba76219687a"} Jul 20 15:51:49.717147 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:49.717058 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9vmxs" event={"ID":"31cf61c8-7ab9-4a0b-9995-be5987ffde36","Type":"ContainerStarted","Data":"f30e51cfaf1761a9221169c501068d97280261f60e8147fabe19235c5dc46aae"} Jul 20 15:51:49.717208 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:49.717141 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:49.719226 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:49.719201 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-frsgc" event={"ID":"f32c9452-5c16-43fd-8612-4c19214966b9","Type":"ContainerStarted","Data":"b5b5de7e96a1aff2a8058b2abc104e58d1f3b9240efbd676d9f9dc20cf01d262"} Jul 20 15:51:49.721366 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:49.721318 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" event={"ID":"a3f580e1-fd44-41dd-8eb0-124be7af2870","Type":"ContainerStarted","Data":"1f664bea08cb960937e08acbc7c9db6e95a7e134f916967acb82c70ea7decfcf"} Jul 20 15:51:49.748133 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:49.747827 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ktzcc" podStartSLOduration=1.865495579 podStartE2EDuration="8.747811421s" totalImagesPullingTime="6.882315842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:49.727200291 +0000 UTC m=+47.905877367" watchObservedRunningTime="2026-07-20 15:51:49.747811421 +0000 UTC m=+47.926488495" Jul 20 15:51:50.878570 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.878524 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9vmxs" podStartSLOduration=3.078196727 podStartE2EDuration="9.878507482s" totalImagesPullingTime="6.800310755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:49.747947154 +0000 UTC m=+47.926624227" watchObservedRunningTime="2026-07-20 15:51:50.878507482 +0000 UTC m=+49.057184536" Jul 20 15:51:50.879044 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.879027 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8c96c586f-4jljc"] Jul 20 15:51:50.896293 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.896271 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c96c586f-4jljc"] Jul 20 15:51:50.896413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.896384 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:50.898750 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.898729 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 15:51:50.898852 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.898792 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jfs8d\"" Jul 20 15:51:50.898852 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.898833 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 15:51:50.899611 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.899595 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 15:51:50.899820 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.899804 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 15:51:50.900034 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.900018 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 15:51:50.946816 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.946791 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-serving-cert\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:50.946952 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.946821 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrc4n\" (UniqueName: \"kubernetes.io/projected/f99e12e1-4ce9-4749-9a13-5984c77e10c8-kube-api-access-hrc4n\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:50.946952 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.946888 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-oauth-serving-cert\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:50.946952 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.946931 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-oauth-config\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:50.947104 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.946979 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-service-ca\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:50.947104 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:50.947025 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-config\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.048271 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.048235 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-config\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.048417 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.048306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-serving-cert\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.048417 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.048331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrc4n\" (UniqueName: \"kubernetes.io/projected/f99e12e1-4ce9-4749-9a13-5984c77e10c8-kube-api-access-hrc4n\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.048417 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.048378 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-oauth-serving-cert\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.048417 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.048414 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-oauth-config\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.048548 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.048476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-service-ca\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.049027 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.049001 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-config\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.049113 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.049069 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-service-ca\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.049622 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.049599 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-oauth-serving-cert\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.052559 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.052537 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-oauth-config\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.052654 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.052559 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-serving-cert\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.056726 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.056705 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrc4n\" (UniqueName: \"kubernetes.io/projected/f99e12e1-4ce9-4749-9a13-5984c77e10c8-kube-api-access-hrc4n\") pod \"console-8c96c586f-4jljc\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.207264 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.207175 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:51:51.730301 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:51.730266 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" event={"ID":"a3f580e1-fd44-41dd-8eb0-124be7af2870","Type":"ContainerStarted","Data":"8002f3dff0f736dbb36c9369e26d4d100d80c8d8f11f3b5a87b871cb32468344"} Jul 20 15:51:53.637103 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:53.637030 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c96c586f-4jljc"] Jul 20 15:51:53.738510 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:53.738470 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" event={"ID":"a3f580e1-fd44-41dd-8eb0-124be7af2870","Type":"ContainerStarted","Data":"9a06494d431ac6a4ad50a1604ad5f2d51ffaa99d767396c1a8733d2b18b69c1a"} Jul 20 15:51:53.758792 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:53.758741 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fhzbr" podStartSLOduration=5.71015952 podStartE2EDuration="7.758528176s" totalImagesPullingTime="2.048368656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:51:53.758201045 +0000 UTC m=+51.936878118" watchObservedRunningTime="2026-07-20 15:51:53.758528176 +0000 UTC m=+51.937205250" Jul 20 15:51:53.781828 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:53.781802 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99e12e1_4ce9_4749_9a13_5984c77e10c8.slice/crio-5c1b437895fec6b56e9a84d3f69e0966317fbde5d07ace68f1a502e2a54b8f2f WatchSource:0}: Error finding container 5c1b437895fec6b56e9a84d3f69e0966317fbde5d07ace68f1a502e2a54b8f2f: Status 404 returned error can't find the container with id 5c1b437895fec6b56e9a84d3f69e0966317fbde5d07ace68f1a502e2a54b8f2f Jul 20 15:51:54.744469 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:54.744430 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c96c586f-4jljc" event={"ID":"f99e12e1-4ce9-4749-9a13-5984c77e10c8","Type":"ContainerStarted","Data":"5c1b437895fec6b56e9a84d3f69e0966317fbde5d07ace68f1a502e2a54b8f2f"} Jul 20 15:51:54.747646 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:54.747584 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-frsgc" event={"ID":"f32c9452-5c16-43fd-8612-4c19214966b9","Type":"ContainerStarted","Data":"32ca003560de41428b876d70a4108610184d575d787e958f60f936c803c5a857"} Jul 20 15:51:55.477163 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.476208 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt"] Jul 20 15:51:55.498917 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.498888 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt"] Jul 20 15:51:55.499075 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.499021 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.503795 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.503769 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 20 15:51:55.504028 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.504011 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 20 15:51:55.504727 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.504544 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-n8tnt\"" Jul 20 15:51:55.518773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.518750 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-ztvqg"] Jul 20 15:51:55.537087 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.537065 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.544390 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.543359 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8spwx"] Jul 20 15:51:55.544732 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.544711 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 20 15:51:55.544838 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.544734 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 20 15:51:55.544838 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.544758 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rv6t7\"" Jul 20 15:51:55.545000 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.544970 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 20 15:51:55.565060 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.565037 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-ztvqg"] Jul 20 15:51:55.565172 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.565148 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.568695 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.568659 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 15:51:55.570523 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.569935 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 15:51:55.570523 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.570255 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cxvjb\"" Jul 20 15:51:55.574394 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.572214 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 15:51:55.584668 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584647 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.584766 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584702 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/151484f3-bc05-40c3-821c-8261ef3e5a3e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.584766 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584720 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.584766 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584735 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/05f0fca8-f08b-4dba-96c4-0995069b8755-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.584928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584770 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.584928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584796 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f0fca8-f08b-4dba-96c4-0995069b8755-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.584928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584816 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.584928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584835 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkn5t\" (UniqueName: \"kubernetes.io/projected/151484f3-bc05-40c3-821c-8261ef3e5a3e-kube-api-access-fkn5t\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.584928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbxct\" (UniqueName: \"kubernetes.io/projected/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-api-access-nbxct\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.584928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.584864 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.686130 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkn5t\" (UniqueName: \"kubernetes.io/projected/151484f3-bc05-40c3-821c-8261ef3e5a3e-kube-api-access-fkn5t\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.686130 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686113 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbxct\" (UniqueName: \"kubernetes.io/projected/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-api-access-nbxct\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.686355 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686141 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.686355 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686169 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-root\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686355 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.686355 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686239 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-tls\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686355 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-wtmp\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686355 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686285 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mssfw\" (UniqueName: \"kubernetes.io/projected/86abd18a-786f-47f4-bf0a-3934f7125d77-kube-api-access-mssfw\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686355 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/151484f3-bc05-40c3-821c-8261ef3e5a3e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/05f0fca8-f08b-4dba-96c4-0995069b8755-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686416 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86abd18a-786f-47f4-bf0a-3934f7125d77-metrics-client-ca\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686472 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-accelerators-collector-config\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686504 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686538 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-sys\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686572 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-textfile\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686599 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686628 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f0fca8-f08b-4dba-96c4-0995069b8755-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.686773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.686713 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.687559 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.687525 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.688237 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:55.687859 2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 20 15:51:55.688237 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:55.687933 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-tls podName:151484f3-bc05-40c3-821c-8261ef3e5a3e nodeName:}" failed. No retries permitted until 2026-07-20 15:51:56.187913524 +0000 UTC m=+54.366590580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-w6crt" (UID: "151484f3-bc05-40c3-821c-8261ef3e5a3e") : secret "openshift-state-metrics-tls" not found Jul 20 15:51:55.688237 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.688182 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/05f0fca8-f08b-4dba-96c4-0995069b8755-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.688537 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.688513 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/151484f3-bc05-40c3-821c-8261ef3e5a3e-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.688617 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.688553 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f0fca8-f08b-4dba-96c4-0995069b8755-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.701146 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.701115 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.701652 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.701523 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.701982 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.701943 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.703875 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.703833 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbxct\" (UniqueName: \"kubernetes.io/projected/05f0fca8-f08b-4dba-96c4-0995069b8755-kube-api-access-nbxct\") pod \"kube-state-metrics-56c5594674-ztvqg\" (UID: \"05f0fca8-f08b-4dba-96c4-0995069b8755\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.705157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.705134 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkn5t\" (UniqueName: \"kubernetes.io/projected/151484f3-bc05-40c3-821c-8261ef3e5a3e-kube-api-access-fkn5t\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:55.787718 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.787664 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-tls\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.787727 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-wtmp\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.787751 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mssfw\" (UniqueName: \"kubernetes.io/projected/86abd18a-786f-47f4-bf0a-3934f7125d77-kube-api-access-mssfw\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.787801 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86abd18a-786f-47f4-bf0a-3934f7125d77-metrics-client-ca\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.787857 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-accelerators-collector-config\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.787890 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-sys\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.787925 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-textfile\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.787954 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.788014 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-root\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788161 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.788097 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-root\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788591 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.788223 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-wtmp\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788591 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.788266 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86abd18a-786f-47f4-bf0a-3934f7125d77-sys\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788591 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.788498 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-textfile\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.788760 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.788632 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-accelerators-collector-config\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.789231 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.789188 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86abd18a-786f-47f4-bf0a-3934f7125d77-metrics-client-ca\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.791805 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.791781 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.793473 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.793452 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86abd18a-786f-47f4-bf0a-3934f7125d77-node-exporter-tls\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.796870 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.796840 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mssfw\" (UniqueName: \"kubernetes.io/projected/86abd18a-786f-47f4-bf0a-3934f7125d77-kube-api-access-mssfw\") pod \"node-exporter-8spwx\" (UID: \"86abd18a-786f-47f4-bf0a-3934f7125d77\") " pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.854774 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.854536 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" Jul 20 15:51:55.883373 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:55.883316 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8spwx" Jul 20 15:51:55.914774 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:55.914741 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86abd18a_786f_47f4_bf0a_3934f7125d77.slice/crio-34d9995bc280f20103ee6e0c14e977cb21475a8f6cf4c236791dec206d45dbad WatchSource:0}: Error finding container 34d9995bc280f20103ee6e0c14e977cb21475a8f6cf4c236791dec206d45dbad: Status 404 returned error can't find the container with id 34d9995bc280f20103ee6e0c14e977cb21475a8f6cf4c236791dec206d45dbad Jul 20 15:51:56.044133 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.044063 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-ztvqg"] Jul 20 15:51:56.047794 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:56.047765 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f0fca8_f08b_4dba_96c4_0995069b8755.slice/crio-49b87489bf58fe0aa6674dc3f271908d2d5aa5b35e8f4e8ee9f7f2f426cde230 WatchSource:0}: Error finding container 49b87489bf58fe0aa6674dc3f271908d2d5aa5b35e8f4e8ee9f7f2f426cde230: Status 404 returned error can't find the container with id 49b87489bf58fe0aa6674dc3f271908d2d5aa5b35e8f4e8ee9f7f2f426cde230 Jul 20 15:51:56.191707 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.191528 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:56.194368 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.194341 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/151484f3-bc05-40c3-821c-8261ef3e5a3e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-w6crt\" (UID: \"151484f3-bc05-40c3-821c-8261ef3e5a3e\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:56.414872 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.414798 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" Jul 20 15:51:56.595293 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.595240 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:51:56.611786 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.611755 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.617785 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.617761 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:51:56.617978 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.617923 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.618180 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.618417 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.618587 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.618751 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.618871 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4lntm\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.619018 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.619151 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.619300 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 15:51:56.619655 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.619532 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 15:51:56.696259 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696189 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696259 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696232 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzqdg\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-kube-api-access-jzqdg\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696458 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696292 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696458 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696341 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-web-config\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696458 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696372 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696458 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696404 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-config-out\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696458 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696429 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696619 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696499 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696619 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696528 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-config-volume\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696619 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696549 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696792 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696614 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696845 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696826 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.696898 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.696857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.755827 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.755796 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" event={"ID":"05f0fca8-f08b-4dba-96c4-0995069b8755","Type":"ContainerStarted","Data":"49b87489bf58fe0aa6674dc3f271908d2d5aa5b35e8f4e8ee9f7f2f426cde230"} Jul 20 15:51:56.757181 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.757136 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8spwx" event={"ID":"86abd18a-786f-47f4-bf0a-3934f7125d77","Type":"ContainerStarted","Data":"34d9995bc280f20103ee6e0c14e977cb21475a8f6cf4c236791dec206d45dbad"} Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.797756 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.797826 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.797858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.797885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.797908 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzqdg\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-kube-api-access-jzqdg\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.797938 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.797965 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-web-config\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.797992 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.798017 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-config-out\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.798031 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.798066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.798107 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-config-volume\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.798134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:56.798497 2559 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.798549 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.798657 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:51:56.798577 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls podName:03980f14-10bc-4759-9f79-7b04a019f044 nodeName:}" failed. No retries permitted until 2026-07-20 15:51:57.298555755 +0000 UTC m=+55.477232811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "03980f14-10bc-4759-9f79-7b04a019f044") : secret "alertmanager-main-tls" not found Jul 20 15:51:56.800092 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.799132 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.801365 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.801337 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.801467 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.801353 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.802395 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.802370 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.802595 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.802570 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.802700 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.802573 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-web-config\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.803091 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.803063 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.804125 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.804099 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-config-volume\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.804224 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.804127 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-config-out\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.804494 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.804451 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:56.807646 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:56.807413 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzqdg\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-kube-api-access-jzqdg\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:57.304802 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.304272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:57.307840 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.307812 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:57.529837 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.529810 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:51:57.616781 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.616704 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78d4478875-n2j2r"] Jul 20 15:51:57.621885 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.621856 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.625399 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.625377 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 20 15:51:57.626015 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.625557 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 20 15:51:57.626015 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.625566 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 20 15:51:57.626015 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.625653 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 20 15:51:57.626015 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.625869 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3pi82bsuksb4t\"" Jul 20 15:51:57.628813 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.628798 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6s8sl\"" Jul 20 15:51:57.629862 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.629761 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 20 15:51:57.645088 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.645067 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78d4478875-n2j2r"] Jul 20 15:51:57.708880 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.708853 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.709027 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.708910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.709027 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.708963 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-grpc-tls\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.709027 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.708991 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp8th\" (UniqueName: \"kubernetes.io/projected/53570cf5-098f-44c5-875f-162878b01ed4-kube-api-access-zp8th\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.709027 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.709023 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-tls\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.709221 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.709106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.709221 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.709136 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.709221 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.709184 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53570cf5-098f-44c5-875f-162878b01ed4-metrics-client-ca\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810072 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810042 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53570cf5-098f-44c5-875f-162878b01ed4-metrics-client-ca\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810489 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810489 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810161 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810622 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-grpc-tls\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810775 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp8th\" (UniqueName: \"kubernetes.io/projected/53570cf5-098f-44c5-875f-162878b01ed4-kube-api-access-zp8th\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810775 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810694 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53570cf5-098f-44c5-875f-162878b01ed4-metrics-client-ca\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810775 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810701 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-tls\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810775 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810770 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.810979 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.810801 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.813954 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.813923 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.815320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.815295 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-tls\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.815856 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.815838 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-grpc-tls\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.816648 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.816614 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.817058 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.817027 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.820015 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.819992 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/53570cf5-098f-44c5-875f-162878b01ed4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.821532 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.821492 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp8th\" (UniqueName: \"kubernetes.io/projected/53570cf5-098f-44c5-875f-162878b01ed4-kube-api-access-zp8th\") pod \"thanos-querier-78d4478875-n2j2r\" (UID: \"53570cf5-098f-44c5-875f-162878b01ed4\") " pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:57.935213 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:57.935149 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:51:59.728308 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.728273 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9vmxs" Jul 20 15:51:59.959403 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.959356 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k"] Jul 20 15:51:59.962521 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.962479 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:51:59.967928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.967451 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 15:51:59.967928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.967508 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 20 15:51:59.967928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.967455 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-63foick4kaqnb\"" Jul 20 15:51:59.967928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.967630 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fhgvj\"" Jul 20 15:51:59.967928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.967788 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 20 15:51:59.967928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.967514 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 20 15:51:59.979188 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.979163 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k"] Jul 20 15:51:59.983780 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:51:59.983666 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:51:59.992803 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:51:59.992771 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03980f14_10bc_4759_9f79_7b04a019f044.slice/crio-be4e5b855622e679778fe685cdff14630732ae1c11b97bc23155dc352c910c7d WatchSource:0}: Error finding container be4e5b855622e679778fe685cdff14630732ae1c11b97bc23155dc352c910c7d: Status 404 returned error can't find the container with id be4e5b855622e679778fe685cdff14630732ae1c11b97bc23155dc352c910c7d Jul 20 15:52:00.011653 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.011610 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt"] Jul 20 15:52:00.019023 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:52:00.018927 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151484f3_bc05_40c3_821c_8261ef3e5a3e.slice/crio-b66c19ece386841eefb32202a8ea069e9c3a746213d13e87e441868463456c98 WatchSource:0}: Error finding container b66c19ece386841eefb32202a8ea069e9c3a746213d13e87e441868463456c98: Status 404 returned error can't find the container with id b66c19ece386841eefb32202a8ea069e9c3a746213d13e87e441868463456c98 Jul 20 15:52:00.032946 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.032913 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78d4478875-n2j2r"] Jul 20 15:52:00.037320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.037176 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e744f9c8-1e9a-4690-a8e3-129270f32547-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.037320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.037206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k59k4\" (UniqueName: \"kubernetes.io/projected/e744f9c8-1e9a-4690-a8e3-129270f32547-kube-api-access-k59k4\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.037453 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.037331 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-client-ca-bundle\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.037453 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.037365 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-secret-metrics-server-tls\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.037453 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.037399 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-secret-metrics-server-client-certs\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.037453 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.037443 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e744f9c8-1e9a-4690-a8e3-129270f32547-metrics-server-audit-profiles\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.037648 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.037498 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e744f9c8-1e9a-4690-a8e3-129270f32547-audit-log\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.041355 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:52:00.041322 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53570cf5_098f_44c5_875f_162878b01ed4.slice/crio-b82e91066266cee69e24744bc9d6fab78c7c99375b0396e4b710d9f6328161d5 WatchSource:0}: Error finding container b82e91066266cee69e24744bc9d6fab78c7c99375b0396e4b710d9f6328161d5: Status 404 returned error can't find the container with id b82e91066266cee69e24744bc9d6fab78c7c99375b0396e4b710d9f6328161d5 Jul 20 15:52:00.139734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.138870 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-client-ca-bundle\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.139734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.138921 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-secret-metrics-server-tls\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.139734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.138952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-secret-metrics-server-client-certs\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.139734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.138984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e744f9c8-1e9a-4690-a8e3-129270f32547-metrics-server-audit-profiles\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.139734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.139025 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e744f9c8-1e9a-4690-a8e3-129270f32547-audit-log\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.139734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.139086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e744f9c8-1e9a-4690-a8e3-129270f32547-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.139734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.139103 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k59k4\" (UniqueName: \"kubernetes.io/projected/e744f9c8-1e9a-4690-a8e3-129270f32547-kube-api-access-k59k4\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.140280 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.140247 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e744f9c8-1e9a-4690-a8e3-129270f32547-metrics-server-audit-profiles\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.140829 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.140774 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e744f9c8-1e9a-4690-a8e3-129270f32547-audit-log\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.141505 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.141479 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e744f9c8-1e9a-4690-a8e3-129270f32547-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.142484 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.142452 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-secret-metrics-server-client-certs\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.144240 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.143321 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-secret-metrics-server-tls\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.145777 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.145743 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e744f9c8-1e9a-4690-a8e3-129270f32547-client-ca-bundle\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.148734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.148663 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k59k4\" (UniqueName: \"kubernetes.io/projected/e744f9c8-1e9a-4690-a8e3-129270f32547-kube-api-access-k59k4\") pod \"metrics-server-6dfd7cb58c-q2m5k\" (UID: \"e744f9c8-1e9a-4690-a8e3-129270f32547\") " pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.235733 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.234716 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf"] Jul 20 15:52:00.237033 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.237014 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:00.239150 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.239125 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w9jrc\"" Jul 20 15:52:00.239295 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.239122 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 20 15:52:00.247247 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.247202 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf"] Jul 20 15:52:00.288599 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.288524 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:00.340100 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.339998 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpf6s\" (UniqueName: \"kubernetes.io/projected/b8613015-77ce-4e72-9894-998c39717652-kube-api-access-dpf6s\") pod \"monitoring-plugin-7b7b7bc8bf-hc4wf\" (UID: \"b8613015-77ce-4e72-9894-998c39717652\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:00.340266 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.340189 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8613015-77ce-4e72-9894-998c39717652-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hc4wf\" (UID: \"b8613015-77ce-4e72-9894-998c39717652\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:00.441398 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.441361 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8613015-77ce-4e72-9894-998c39717652-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hc4wf\" (UID: \"b8613015-77ce-4e72-9894-998c39717652\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:00.441577 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.441435 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpf6s\" (UniqueName: \"kubernetes.io/projected/b8613015-77ce-4e72-9894-998c39717652-kube-api-access-dpf6s\") pod \"monitoring-plugin-7b7b7bc8bf-hc4wf\" (UID: \"b8613015-77ce-4e72-9894-998c39717652\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:00.441897 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:52:00.441877 2559 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 20 15:52:00.441984 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:52:00.441947 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8613015-77ce-4e72-9894-998c39717652-monitoring-plugin-cert podName:b8613015-77ce-4e72-9894-998c39717652 nodeName:}" failed. No retries permitted until 2026-07-20 15:52:00.941926307 +0000 UTC m=+59.120603364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b8613015-77ce-4e72-9894-998c39717652-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-hc4wf" (UID: "b8613015-77ce-4e72-9894-998c39717652") : secret "monitoring-plugin-cert" not found Jul 20 15:52:00.450040 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.449975 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpf6s\" (UniqueName: \"kubernetes.io/projected/b8613015-77ce-4e72-9894-998c39717652-kube-api-access-dpf6s\") pod \"monitoring-plugin-7b7b7bc8bf-hc4wf\" (UID: \"b8613015-77ce-4e72-9894-998c39717652\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:00.453374 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.453338 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k"] Jul 20 15:52:00.635071 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.634991 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-b675c7cd7-shkfn"] Jul 20 15:52:00.638662 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.638627 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.642956 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.642931 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 20 15:52:00.643115 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.642999 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 20 15:52:00.643336 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.643312 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5t4fb\"" Jul 20 15:52:00.643511 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.643492 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 20 15:52:00.643709 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.643667 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 20 15:52:00.643830 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.643809 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 20 15:52:00.650418 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.650181 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 20 15:52:00.655467 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.655374 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b675c7cd7-shkfn"] Jul 20 15:52:00.717538 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:52:00.717499 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode744f9c8_1e9a_4690_a8e3_129270f32547.slice/crio-e50dac68f7557e3b5b8b3ef77ff8f01086b484f997d99c3eb157099f9a5086eb WatchSource:0}: Error finding container e50dac68f7557e3b5b8b3ef77ff8f01086b484f997d99c3eb157099f9a5086eb: Status 404 returned error can't find the container with id e50dac68f7557e3b5b8b3ef77ff8f01086b484f997d99c3eb157099f9a5086eb Jul 20 15:52:00.744361 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.744321 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-secret-telemeter-client\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.744819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.744435 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-federate-client-tls\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.744819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.744487 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4dvk\" (UniqueName: \"kubernetes.io/projected/ec87b338-81e2-4a67-8265-217a7dc32157-kube-api-access-w4dvk\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.744819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.744540 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-serving-certs-ca-bundle\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.744819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.744584 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-metrics-client-ca\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.744819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.744624 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.744819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.744726 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.744819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.744776 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-telemeter-client-tls\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.776340 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.776247 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-frsgc" event={"ID":"f32c9452-5c16-43fd-8612-4c19214966b9","Type":"ContainerStarted","Data":"2e0a132fab69268a07ebc9f2bb52015e52e3a18f0ba13d593d4df563b3573f2c"} Jul 20 15:52:00.778129 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.778101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" event={"ID":"e744f9c8-1e9a-4690-a8e3-129270f32547","Type":"ContainerStarted","Data":"e50dac68f7557e3b5b8b3ef77ff8f01086b484f997d99c3eb157099f9a5086eb"} Jul 20 15:52:00.779910 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.779874 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" event={"ID":"53570cf5-098f-44c5-875f-162878b01ed4","Type":"ContainerStarted","Data":"b82e91066266cee69e24744bc9d6fab78c7c99375b0396e4b710d9f6328161d5"} Jul 20 15:52:00.781567 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.781539 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerStarted","Data":"be4e5b855622e679778fe685cdff14630732ae1c11b97bc23155dc352c910c7d"} Jul 20 15:52:00.783256 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.783219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c96c586f-4jljc" event={"ID":"f99e12e1-4ce9-4749-9a13-5984c77e10c8","Type":"ContainerStarted","Data":"6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4"} Jul 20 15:52:00.784759 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.784730 2559 generic.go:356] "Generic (PLEG): container finished" podID="86abd18a-786f-47f4-bf0a-3934f7125d77" containerID="b2f6a5f12360fc0da239fac2a37c7bd9e5409b4f4ab4fe174aeafc5ec3e03682" exitCode=0 Jul 20 15:52:00.784883 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.784807 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8spwx" event={"ID":"86abd18a-786f-47f4-bf0a-3934f7125d77","Type":"ContainerDied","Data":"b2f6a5f12360fc0da239fac2a37c7bd9e5409b4f4ab4fe174aeafc5ec3e03682"} Jul 20 15:52:00.786605 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.786571 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" event={"ID":"151484f3-bc05-40c3-821c-8261ef3e5a3e","Type":"ContainerStarted","Data":"07afd7a720f58bdc0df1c8b7b04ea9b6977e21834c97081f0d5be94b7045448f"} Jul 20 15:52:00.786705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.786633 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" event={"ID":"151484f3-bc05-40c3-821c-8261ef3e5a3e","Type":"ContainerStarted","Data":"89eb96e5495390c5ca5cd5f713ae22560d863a2d5dcec7b5531d0ba8f5887557"} Jul 20 15:52:00.786705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.786664 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" event={"ID":"151484f3-bc05-40c3-821c-8261ef3e5a3e","Type":"ContainerStarted","Data":"b66c19ece386841eefb32202a8ea069e9c3a746213d13e87e441868463456c98"} Jul 20 15:52:00.796710 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.796593 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-frsgc" podStartSLOduration=2.002263781 podStartE2EDuration="19.796571956s" totalImagesPullingTime="17.794308175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:00.794852522 +0000 UTC m=+58.973529608" watchObservedRunningTime="2026-07-20 15:52:00.796571956 +0000 UTC m=+58.975249032" Jul 20 15:52:00.812446 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.812388 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8c96c586f-4jljc" podStartSLOduration=4.847374059 podStartE2EDuration="10.812365355s" totalImagesPullingTime="5.964991296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:00.811201294 +0000 UTC m=+58.989878369" watchObservedRunningTime="2026-07-20 15:52:00.812365355 +0000 UTC m=+58.991042429" Jul 20 15:52:00.846730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.845230 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-secret-telemeter-client\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.846730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.845380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-federate-client-tls\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.846730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.845414 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4dvk\" (UniqueName: \"kubernetes.io/projected/ec87b338-81e2-4a67-8265-217a7dc32157-kube-api-access-w4dvk\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.846730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.845431 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-serving-certs-ca-bundle\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.846730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.845446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-metrics-client-ca\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.846730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.845469 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.846730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.845505 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.846730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.845543 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-telemeter-client-tls\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.847197 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.847061 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-serving-certs-ca-bundle\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.849370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.848356 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-metrics-client-ca\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.849370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.849318 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec87b338-81e2-4a67-8265-217a7dc32157-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.851628 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.851417 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-secret-telemeter-client\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.851628 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.851534 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-federate-client-tls\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.851628 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.851561 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-telemeter-client-tls\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.853114 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.853041 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec87b338-81e2-4a67-8265-217a7dc32157-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.857469 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.857145 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4dvk\" (UniqueName: \"kubernetes.io/projected/ec87b338-81e2-4a67-8265-217a7dc32157-kube-api-access-w4dvk\") pod \"telemeter-client-b675c7cd7-shkfn\" (UID: \"ec87b338-81e2-4a67-8265-217a7dc32157\") " pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.947065 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.946831 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8613015-77ce-4e72-9894-998c39717652-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hc4wf\" (UID: \"b8613015-77ce-4e72-9894-998c39717652\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:00.951658 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.951608 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" Jul 20 15:52:00.959226 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:00.959184 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8613015-77ce-4e72-9894-998c39717652-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hc4wf\" (UID: \"b8613015-77ce-4e72-9894-998c39717652\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:01.158826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.151311 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:01.211710 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.210656 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:52:01.211710 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.210819 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:52:01.213135 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.212985 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b675c7cd7-shkfn"] Jul 20 15:52:01.217861 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:52:01.217819 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec87b338_81e2_4a67_8265_217a7dc32157.slice/crio-6c12bfcb39f90baae91451cd2bd290563c7244ecc76bb342fc55c397716e79f4 WatchSource:0}: Error finding container 6c12bfcb39f90baae91451cd2bd290563c7244ecc76bb342fc55c397716e79f4: Status 404 returned error can't find the container with id 6c12bfcb39f90baae91451cd2bd290563c7244ecc76bb342fc55c397716e79f4 Jul 20 15:52:01.222786 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.222494 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:52:01.269416 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.269376 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dbf99f667-bhtq5"] Jul 20 15:52:01.276632 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.275851 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.285699 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.283741 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dbf99f667-bhtq5"] Jul 20 15:52:01.287529 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.287502 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 15:52:01.356979 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.356395 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-oauth-serving-cert\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.356979 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.356457 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-oauth-config\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.356979 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.356552 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-trusted-ca-bundle\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.356979 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.356631 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-config\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.356979 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.356665 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-serving-cert\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.356979 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.356713 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp5fg\" (UniqueName: \"kubernetes.io/projected/f838e05a-0aaf-469f-aa76-40d11c33d08f-kube-api-access-fp5fg\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.356979 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.356786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-service-ca\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.373805 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.373726 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf"] Jul 20 15:52:01.458283 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.458201 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-service-ca\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.458465 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.458283 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-oauth-serving-cert\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.458465 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.458321 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-oauth-config\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.458465 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.458365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-trusted-ca-bundle\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.458465 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.458435 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-config\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.458703 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.458467 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-serving-cert\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.458703 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.458489 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp5fg\" (UniqueName: \"kubernetes.io/projected/f838e05a-0aaf-469f-aa76-40d11c33d08f-kube-api-access-fp5fg\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.459636 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.459607 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-service-ca\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.460230 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.460207 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-oauth-serving-cert\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.461464 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.461152 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-config\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.461464 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.461416 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-trusted-ca-bundle\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.465259 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.465206 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-serving-cert\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.467299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.467214 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-oauth-config\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.470999 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.470966 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp5fg\" (UniqueName: \"kubernetes.io/projected/f838e05a-0aaf-469f-aa76-40d11c33d08f-kube-api-access-fp5fg\") pod \"console-6dbf99f667-bhtq5\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.599474 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.599215 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:01.795232 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.795142 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" event={"ID":"b8613015-77ce-4e72-9894-998c39717652","Type":"ContainerStarted","Data":"707b5f4261084bc8c2519a0a337a9ad3b80e12a842f88a2aece244f9d0aed216"} Jul 20 15:52:01.801506 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.801465 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" event={"ID":"05f0fca8-f08b-4dba-96c4-0995069b8755","Type":"ContainerStarted","Data":"c06ac305b1d32487c153972e632660bb19fd4ac8d1511b36479dc31faa8c203e"} Jul 20 15:52:01.801726 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.801543 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" event={"ID":"05f0fca8-f08b-4dba-96c4-0995069b8755","Type":"ContainerStarted","Data":"f2082041d03adac9239c0814e4ea1dad2e9f4b5ed236cdd1be3014cac3b4ea25"} Jul 20 15:52:01.801726 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.801576 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" event={"ID":"05f0fca8-f08b-4dba-96c4-0995069b8755","Type":"ContainerStarted","Data":"b8218d2654253bc969c6c31c3706203093a850d7341b95c92927a0e841ac1106"} Jul 20 15:52:01.803095 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.803006 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" event={"ID":"ec87b338-81e2-4a67-8265-217a7dc32157","Type":"ContainerStarted","Data":"6c12bfcb39f90baae91451cd2bd290563c7244ecc76bb342fc55c397716e79f4"} Jul 20 15:52:01.807757 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.807723 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8spwx" event={"ID":"86abd18a-786f-47f4-bf0a-3934f7125d77","Type":"ContainerStarted","Data":"8891b0e91561b2f1d01629c12b871329c73070293c40275cba53051c14ef20a4"} Jul 20 15:52:01.807912 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.807783 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8spwx" event={"ID":"86abd18a-786f-47f4-bf0a-3934f7125d77","Type":"ContainerStarted","Data":"1e7ba7d0783fdc6a204124525d2c426e134d5868840c4fc8550c9a295d5a6398"} Jul 20 15:52:01.816703 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.813835 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:52:01.823997 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.823735 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-ztvqg" podStartSLOduration=2.11823201 podStartE2EDuration="6.823718153s" totalImagesPullingTime="4.705486143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:01.821531469 +0000 UTC m=+60.000208544" watchObservedRunningTime="2026-07-20 15:52:01.823718153 +0000 UTC m=+60.002395218" Jul 20 15:52:01.851359 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.851317 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:52:01.854972 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.854944 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.863408 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.860945 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 20 15:52:01.863408 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.861021 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 20 15:52:01.863408 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.860956 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 20 15:52:01.867051 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.865812 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 20 15:52:01.867051 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.866162 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 20 15:52:01.867051 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.866173 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 20 15:52:01.867051 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.866404 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bwspc\"" Jul 20 15:52:01.867051 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.866616 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 20 15:52:01.867051 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.866836 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 20 15:52:01.869204 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.867731 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 20 15:52:01.872021 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.870353 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 20 15:52:01.872021 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.870562 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 20 15:52:01.872021 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.870838 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 20 15:52:01.872021 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.871036 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 20 15:52:01.872021 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.871218 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dq57aocrjctde\"" Jul 20 15:52:01.877121 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.877091 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 20 15:52:01.880598 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.880549 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:52:01.892575 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.892424 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8spwx" podStartSLOduration=2.892036381 podStartE2EDuration="6.892401901s" totalImagesPullingTime="3.904024832s" totalInitContainerRuntime="96.340688ms" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:01.888002874 +0000 UTC m=+60.066679948" watchObservedRunningTime="2026-07-20 15:52:01.892401901 +0000 UTC m=+60.071078975" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.965994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966063 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966080 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966177 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-web-config\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966205 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-config-out\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966221 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-config\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966239 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966289 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966351 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966429 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.966567 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.967650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.967574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.968237 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.967790 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.968237 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.967830 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.968237 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.967866 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.968237 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.967909 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.968237 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.967942 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.968237 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.967966 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ndnm\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-kube-api-access-7ndnm\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:01.968237 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:01.968207 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069147 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069055 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069147 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069111 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069147 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069142 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069445 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069168 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069445 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069192 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069445 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069210 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ndnm\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-kube-api-access-7ndnm\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069445 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069236 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069445 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069295 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069445 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069348 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069445 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069373 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069445 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069443 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-web-config\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069477 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-config-out\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069504 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-config\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069526 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069551 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069582 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069630 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069720 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.069881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.069782 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.073202 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.070603 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.073202 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.071377 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.073202 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.071507 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.073202 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.072528 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.073202 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.072626 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.073202 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.073053 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.073697 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.073279 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.077734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.077566 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.077885 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.077732 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.078024 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.078002 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.082907 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.079946 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.082907 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.080094 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.082907 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.081594 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.085667 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.085182 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-config-out\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.085667 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.085245 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.085667 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.085629 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.085916 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.085740 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-web-config\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.086116 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.086090 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.087507 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.087466 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ndnm\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-kube-api-access-7ndnm\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.087765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.087721 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-config\") pod \"prometheus-k8s-0\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:02.174489 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.174220 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bwspc\"" Jul 20 15:52:02.182392 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:02.182177 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:03.637584 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:03.637546 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6686p" Jul 20 15:52:08.035636 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.035591 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:52:08.036193 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.035735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:52:08.038442 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.038418 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 15:52:08.038841 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.038809 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 15:52:08.049265 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.049234 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3d9ab51-3b57-4749-9de3-4724174739f4-original-pull-secret\") pod \"global-pull-secret-syncer-kjkb7\" (UID: \"c3d9ab51-3b57-4749-9de3-4724174739f4\") " pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:52:08.049490 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.049245 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8bc050-0d54-4bf9-977f-3496292834bd-metrics-certs\") pod \"network-metrics-daemon-l9db4\" (UID: \"9e8bc050-0d54-4bf9-977f-3496292834bd\") " pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:52:08.137113 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.137071 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:52:08.140099 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.140064 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 15:52:08.145090 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.145059 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7vt6p\"" Jul 20 15:52:08.149661 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.149634 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 15:52:08.151755 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.151729 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9db4" Jul 20 15:52:08.161333 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.161292 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v2ts\" (UniqueName: \"kubernetes.io/projected/ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a-kube-api-access-8v2ts\") pod \"network-check-target-gxvpj\" (UID: \"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a\") " pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:52:08.320882 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.320800 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kjkb7" Jul 20 15:52:08.436793 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.436764 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6j7cm\"" Jul 20 15:52:08.445718 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:08.445666 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:52:12.142370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:12.142336 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dbf99f667-bhtq5"] Jul 20 15:52:12.563921 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:52:12.563883 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf838e05a_0aaf_469f_aa76_40d11c33d08f.slice/crio-d6af57ee9571d1e2a84949ad0da1bb774d0dd438323345523bbdd7d925c21aba WatchSource:0}: Error finding container d6af57ee9571d1e2a84949ad0da1bb774d0dd438323345523bbdd7d925c21aba: Status 404 returned error can't find the container with id d6af57ee9571d1e2a84949ad0da1bb774d0dd438323345523bbdd7d925c21aba Jul 20 15:52:12.717000 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:12.716967 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:52:12.852694 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:12.852599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbf99f667-bhtq5" event={"ID":"f838e05a-0aaf-469f-aa76-40d11c33d08f","Type":"ContainerStarted","Data":"d6af57ee9571d1e2a84949ad0da1bb774d0dd438323345523bbdd7d925c21aba"} Jul 20 15:52:13.281628 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.281597 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kjkb7"] Jul 20 15:52:13.298141 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:52:13.298105 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d9ab51_3b57_4749_9de3_4724174739f4.slice/crio-ef141da7ef92e93989c0c84a91325cff7b1bceeb62a324dd466bfcd7f6302bc9 WatchSource:0}: Error finding container ef141da7ef92e93989c0c84a91325cff7b1bceeb62a324dd466bfcd7f6302bc9: Status 404 returned error can't find the container with id ef141da7ef92e93989c0c84a91325cff7b1bceeb62a324dd466bfcd7f6302bc9 Jul 20 15:52:13.304174 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.304151 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gxvpj"] Jul 20 15:52:13.306881 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:52:13.306856 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea9afdcf_d8af_46d9_9a3c_a51008b6ec8a.slice/crio-be7a8de76f1fc3bbb1c23b67b576fb7aa57b92d289ac2c0b43e961ea2bd4ee12 WatchSource:0}: Error finding container be7a8de76f1fc3bbb1c23b67b576fb7aa57b92d289ac2c0b43e961ea2bd4ee12: Status 404 returned error can't find the container with id be7a8de76f1fc3bbb1c23b67b576fb7aa57b92d289ac2c0b43e961ea2bd4ee12 Jul 20 15:52:13.317154 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.317118 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9db4"] Jul 20 15:52:13.320300 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:52:13.320278 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8bc050_0d54_4bf9_977f_3496292834bd.slice/crio-e196fb319d36ad80836e1b110c8ba76203bfb412aa5a23d63aac7e938984e08c WatchSource:0}: Error finding container e196fb319d36ad80836e1b110c8ba76203bfb412aa5a23d63aac7e938984e08c: Status 404 returned error can't find the container with id e196fb319d36ad80836e1b110c8ba76203bfb412aa5a23d63aac7e938984e08c Jul 20 15:52:13.865923 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.864721 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" event={"ID":"b8613015-77ce-4e72-9894-998c39717652","Type":"ContainerStarted","Data":"50cd32ccb1086dc09795c57598cdf47859adf01d4842e93693e0fcc80e899466"} Jul 20 15:52:13.865923 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.865822 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:13.870432 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.870395 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" event={"ID":"e744f9c8-1e9a-4690-a8e3-129270f32547","Type":"ContainerStarted","Data":"0f60bedc557fac1166eeb19550732569db1db231d6b97cc206612e3575c302f1"} Jul 20 15:52:13.872955 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.872930 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" Jul 20 15:52:13.875850 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.875827 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" event={"ID":"53570cf5-098f-44c5-875f-162878b01ed4","Type":"ContainerStarted","Data":"af0e07e2c0ca016e2eee6564eaaef978201c814588481ac09327c914794fa497"} Jul 20 15:52:13.875850 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.875877 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" event={"ID":"53570cf5-098f-44c5-875f-162878b01ed4","Type":"ContainerStarted","Data":"ea8794639a245bd15ae56b519a52bf1bc2d4735f2168cf33131983e83c5ee64a"} Jul 20 15:52:13.875850 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.875904 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" event={"ID":"53570cf5-098f-44c5-875f-162878b01ed4","Type":"ContainerStarted","Data":"71aa2d02e85a78a7314439a1e1fbc6cb975ac61857070493d42d23d2dcef9cb2"} Jul 20 15:52:13.878693 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.878259 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbf99f667-bhtq5" event={"ID":"f838e05a-0aaf-469f-aa76-40d11c33d08f","Type":"ContainerStarted","Data":"c6b53d4a9e49eaf9e7811b2e8d779de61479ec178b2430e120822932e9110415"} Jul 20 15:52:13.883652 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.881486 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-fnnpd" event={"ID":"cbf42dd3-8ef3-48b3-86f7-5a66f14814dc","Type":"ContainerStarted","Data":"e285a6b49f08ed5dc5d6533ca64228e9c379038f35425d9d0f0d4830c04bf42e"} Jul 20 15:52:13.883652 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.882193 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-fnnpd" Jul 20 15:52:13.885351 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.885262 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gxvpj" event={"ID":"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a","Type":"ContainerStarted","Data":"be7a8de76f1fc3bbb1c23b67b576fb7aa57b92d289ac2c0b43e961ea2bd4ee12"} Jul 20 15:52:13.888952 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.888474 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hc4wf" podStartSLOduration=1.9477521169999998 podStartE2EDuration="13.888438666s" totalImagesPullingTime="11.940686549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:13.887553997 +0000 UTC m=+72.066231069" watchObservedRunningTime="2026-07-20 15:52:13.888438666 +0000 UTC m=+72.067115720" Jul 20 15:52:13.888952 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.888533 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kjkb7" event={"ID":"c3d9ab51-3b57-4749-9de3-4724174739f4","Type":"ContainerStarted","Data":"ef141da7ef92e93989c0c84a91325cff7b1bceeb62a324dd466bfcd7f6302bc9"} Jul 20 15:52:13.895831 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.895022 2559 generic.go:356] "Generic (PLEG): container finished" podID="03980f14-10bc-4759-9f79-7b04a019f044" containerID="27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8" exitCode=0 Jul 20 15:52:13.895831 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.895093 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerDied","Data":"27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8"} Jul 20 15:52:13.895831 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.895437 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-fnnpd" Jul 20 15:52:13.899967 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.899943 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" event={"ID":"ec87b338-81e2-4a67-8265-217a7dc32157","Type":"ContainerStarted","Data":"2b9758fdc3d875099751025d3899c091b623f1a47077855cabab7a878cfea377"} Jul 20 15:52:13.900048 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.899995 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" event={"ID":"ec87b338-81e2-4a67-8265-217a7dc32157","Type":"ContainerStarted","Data":"aab177523732df71c447eb349c35060cbf4256a63f35fb56af55c17de11cbc9d"} Jul 20 15:52:13.900048 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.900023 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" event={"ID":"ec87b338-81e2-4a67-8265-217a7dc32157","Type":"ContainerStarted","Data":"e9e526bd496789815ab4c1e0f89b2533e50612f78a541767e4574aca96519101"} Jul 20 15:52:13.901888 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.901839 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9db4" event={"ID":"9e8bc050-0d54-4bf9-977f-3496292834bd","Type":"ContainerStarted","Data":"e196fb319d36ad80836e1b110c8ba76203bfb412aa5a23d63aac7e938984e08c"} Jul 20 15:52:13.908529 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.908500 2559 generic.go:356] "Generic (PLEG): container finished" podID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerID="bc369aa64f7c5a24ec2e89c349accf4911513f9ef041b6ecfa778ab2f64b0526" exitCode=0 Jul 20 15:52:13.908620 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.908595 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerDied","Data":"bc369aa64f7c5a24ec2e89c349accf4911513f9ef041b6ecfa778ab2f64b0526"} Jul 20 15:52:13.908753 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.908655 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerStarted","Data":"b1846d70a3ccd2bae5d38109c0a6331e23c51587ff225ba33e2234f4396054ac"} Jul 20 15:52:13.916251 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.915718 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" event={"ID":"151484f3-bc05-40c3-821c-8261ef3e5a3e","Type":"ContainerStarted","Data":"7175753a2dd351ce5567ec5f69e4c4dfea5ddb51e820818b1c7da0701249e644"} Jul 20 15:52:13.941845 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:13.941206 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" podStartSLOduration=2.337614544 podStartE2EDuration="14.941183736s" totalImagesPullingTime="12.603569192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:13.940791371 +0000 UTC m=+72.119468447" watchObservedRunningTime="2026-07-20 15:52:13.941183736 +0000 UTC m=+72.119860792" Jul 20 15:52:14.021109 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:14.021050 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-fnnpd" podStartSLOduration=6.291334494 podStartE2EDuration="30.021028322s" totalImagesPullingTime="23.729693828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:13.972074397 +0000 UTC m=+72.150751471" watchObservedRunningTime="2026-07-20 15:52:14.021028322 +0000 UTC m=+72.199705397" Jul 20 15:52:14.022324 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:14.022275 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dbf99f667-bhtq5" podStartSLOduration=13.022261871 podStartE2EDuration="13.022261871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:14.013885115 +0000 UTC m=+72.192562189" watchObservedRunningTime="2026-07-20 15:52:14.022261871 +0000 UTC m=+72.200938948" Jul 20 15:52:14.112068 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:14.111350 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-b675c7cd7-shkfn" podStartSLOduration=2.008000216 podStartE2EDuration="14.111301846s" totalImagesPullingTime="12.10330163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:14.071949809 +0000 UTC m=+72.250626883" watchObservedRunningTime="2026-07-20 15:52:14.111301846 +0000 UTC m=+72.289978920" Jul 20 15:52:14.164816 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:14.164702 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-w6crt" podStartSLOduration=6.057181467 podStartE2EDuration="19.164665275s" totalImagesPullingTime="13.107483808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:14.163338378 +0000 UTC m=+72.342015454" watchObservedRunningTime="2026-07-20 15:52:14.164665275 +0000 UTC m=+72.343342349" Jul 20 15:52:15.851871 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.851835 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dbf99f667-bhtq5"] Jul 20 15:52:15.886738 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.885065 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-584dcbdb6b-hxgmc"] Jul 20 15:52:15.900831 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.900805 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-584dcbdb6b-hxgmc"] Jul 20 15:52:15.900982 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.900914 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:15.937446 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.937010 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctp7h\" (UniqueName: \"kubernetes.io/projected/921105af-900c-451b-9446-01ad3d088ccb-kube-api-access-ctp7h\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:15.937446 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.937087 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-serving-cert\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:15.937446 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.937117 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-trusted-ca-bundle\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:15.937446 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.937205 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-oauth-config\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:15.937446 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.937235 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-service-ca\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:15.937446 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.937282 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-console-config\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:15.937446 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:15.937326 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-oauth-serving-cert\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.037893 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-console-config\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.037973 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-oauth-serving-cert\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.038008 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctp7h\" (UniqueName: \"kubernetes.io/projected/921105af-900c-451b-9446-01ad3d088ccb-kube-api-access-ctp7h\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.038065 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-serving-cert\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.038090 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-trusted-ca-bundle\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.038181 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-oauth-config\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.038209 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-service-ca\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.038999 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-service-ca\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.039791 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-console-config\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.040705 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.040636 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-oauth-serving-cert\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.042808 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.042777 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-trusted-ca-bundle\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.051571 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.051531 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctp7h\" (UniqueName: \"kubernetes.io/projected/921105af-900c-451b-9446-01ad3d088ccb-kube-api-access-ctp7h\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.051811 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.051745 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-oauth-config\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.057494 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.057450 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-serving-cert\") pod \"console-584dcbdb6b-hxgmc\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:16.215600 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:16.215287 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:20.289256 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:20.289182 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:20.290052 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:20.289975 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:21.600802 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:21.600765 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:25.717409 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:25.716500 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-584dcbdb6b-hxgmc"] Jul 20 15:52:26.001198 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.001047 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" event={"ID":"53570cf5-098f-44c5-875f-162878b01ed4","Type":"ContainerStarted","Data":"a2502aee23f1764cdb1d3eaafde7374d86c940779ddd0b4b4e311ea90d61aa36"} Jul 20 15:52:26.001198 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.001131 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" event={"ID":"53570cf5-098f-44c5-875f-162878b01ed4","Type":"ContainerStarted","Data":"86789f52fe161293047aaa3b4dcac66843401f0422ab90b3138dab7ae260da69"} Jul 20 15:52:26.010030 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.008804 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gxvpj" event={"ID":"ea9afdcf-d8af-46d9-9a3c-a51008b6ec8a","Type":"ContainerStarted","Data":"acf2139fc0bb56980132209ec9312374885ac7310b92d3bc093e240103a0e0f7"} Jul 20 15:52:26.010030 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.009188 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:52:26.014397 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.014367 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kjkb7" event={"ID":"c3d9ab51-3b57-4749-9de3-4724174739f4","Type":"ContainerStarted","Data":"99f6c82fbc1a4245f1c4dc899bb5e5cff876e94e71bc7d328f617c82b5314f0d"} Jul 20 15:52:26.019743 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.019498 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584dcbdb6b-hxgmc" event={"ID":"921105af-900c-451b-9446-01ad3d088ccb","Type":"ContainerStarted","Data":"ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7"} Jul 20 15:52:26.019743 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.019547 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584dcbdb6b-hxgmc" event={"ID":"921105af-900c-451b-9446-01ad3d088ccb","Type":"ContainerStarted","Data":"32bb85d932219878839b27fb55941a4689401b0c69fc7fec9c7564a7e53ad94c"} Jul 20 15:52:26.030443 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.030343 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerStarted","Data":"8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd"} Jul 20 15:52:26.038266 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.035596 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gxvpj" podStartSLOduration=70.73415372 podStartE2EDuration="1m23.035579916s" totalImagesPullingTime="12.301426196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:26.034635249 +0000 UTC m=+84.213312322" watchObservedRunningTime="2026-07-20 15:52:26.035579916 +0000 UTC m=+84.214256988" Jul 20 15:52:26.038266 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.036564 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9db4" event={"ID":"9e8bc050-0d54-4bf9-977f-3496292834bd","Type":"ContainerStarted","Data":"2d0f06324f0f9c44333d207646ee99b8331ccaff787ec6060874e08c07c3a124"} Jul 20 15:52:26.038266 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.036615 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9db4" event={"ID":"9e8bc050-0d54-4bf9-977f-3496292834bd","Type":"ContainerStarted","Data":"110bd51dad19f6b9b05726b2a8d26a46560c8c575087b4628c97a8f25084fba0"} Jul 20 15:52:26.040138 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.039962 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerStarted","Data":"95b853f8259c101d51e231259645cdc2ffb1874b26ee0aadf984745ec5aa6af2"} Jul 20 15:52:26.040138 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.040028 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerStarted","Data":"360c126beb5410f524dbbe33b0d486556096284977065e344a06535c9b97bb03"} Jul 20 15:52:26.060646 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.059825 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-584dcbdb6b-hxgmc" podStartSLOduration=11.059804157 podStartE2EDuration="11.059804157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:52:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:26.057260474 +0000 UTC m=+84.235937540" watchObservedRunningTime="2026-07-20 15:52:26.059804157 +0000 UTC m=+84.238481231" Jul 20 15:52:26.075698 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.075634 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kjkb7" podStartSLOduration=70.75222458 podStartE2EDuration="1m23.07561341s" totalImagesPullingTime="12.32338883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:26.074335996 +0000 UTC m=+84.253013074" watchObservedRunningTime="2026-07-20 15:52:26.07561341 +0000 UTC m=+84.254290485" Jul 20 15:52:26.216199 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.216117 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:26.216199 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.216172 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:26.221088 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.221066 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:26.254320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:26.252889 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l9db4" podStartSLOduration=75.346496164 podStartE2EDuration="1m23.252869029s" totalImagesPullingTime="7.906372865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:26.101563167 +0000 UTC m=+84.280240241" watchObservedRunningTime="2026-07-20 15:52:26.252869029 +0000 UTC m=+84.431546102" Jul 20 15:52:27.048189 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.048135 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" event={"ID":"53570cf5-098f-44c5-875f-162878b01ed4","Type":"ContainerStarted","Data":"b252edd31d95559030d5dc1207ccfc636d5d66512637d91ebafe2f2614023d98"} Jul 20 15:52:27.048776 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.048236 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:52:27.052164 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.052128 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerStarted","Data":"52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908"} Jul 20 15:52:27.052352 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.052215 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerStarted","Data":"2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9"} Jul 20 15:52:27.052352 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.052260 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerStarted","Data":"37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4"} Jul 20 15:52:27.052352 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.052289 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerStarted","Data":"8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc"} Jul 20 15:52:27.052352 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.052321 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerStarted","Data":"1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5"} Jul 20 15:52:27.055066 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.055042 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" Jul 20 15:52:27.056175 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.056140 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerStarted","Data":"34ca26cd604b17f78eea2026cb4065d7f44b2d3e7ee0dd16d940c36e04b2d349"} Jul 20 15:52:27.056334 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.056220 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerStarted","Data":"1d8aeadb7533371e8c37789e1a32f0b7b3d726dcb612de821e84019ee36623b4"} Jul 20 15:52:27.056334 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.056259 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerStarted","Data":"0961be1f3e4832ed77dc8641799addc78af686b5c11f155d9e41f96d27c0ebb3"} Jul 20 15:52:27.056334 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.056298 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerStarted","Data":"f6f164657b70f740290b989d24f7962b69cbb5b05272265e5c3008e9441b1be4"} Jul 20 15:52:27.060133 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.060106 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:52:27.121912 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.121852 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78d4478875-n2j2r" podStartSLOduration=5.047534646 podStartE2EDuration="30.121831703s" totalImagesPullingTime="25.074297057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:27.086290999 +0000 UTC m=+85.264968072" watchObservedRunningTime="2026-07-20 15:52:27.121831703 +0000 UTC m=+85.300508775" Jul 20 15:52:27.154426 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.154361 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.094343946 podStartE2EDuration="31.154336016s" totalImagesPullingTime="25.040024695s" totalInitContainerRuntime="19.967375ms" isStatefulPod=false podCreationTimestamp="2026-07-20 15:51:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:27.152035368 +0000 UTC m=+85.330712453" watchObservedRunningTime="2026-07-20 15:52:27.154336016 +0000 UTC m=+85.333013089" Jul 20 15:52:27.182887 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.182854 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:52:27.223335 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.223283 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=14.441838191 podStartE2EDuration="26.223263422s" totalImagesPullingTime="11.711168278s" totalInitContainerRuntime="70.256953ms" isStatefulPod=false podCreationTimestamp="2026-07-20 15:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:27.22184455 +0000 UTC m=+85.400521634" watchObservedRunningTime="2026-07-20 15:52:27.223263422 +0000 UTC m=+85.401940494" Jul 20 15:52:27.245930 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:27.245890 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c96c586f-4jljc"] Jul 20 15:52:40.294530 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:40.294501 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:40.300077 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:40.300033 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6dfd7cb58c-q2m5k" Jul 20 15:52:40.968763 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:40.968608 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6dbf99f667-bhtq5" podUID="f838e05a-0aaf-469f-aa76-40d11c33d08f" containerName="console" containerID="cri-o://c6b53d4a9e49eaf9e7811b2e8d779de61479ec178b2430e120822932e9110415" gracePeriod=15 Jul 20 15:52:41.138560 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.138522 2559 generic.go:356] "Generic (PLEG): container finished" podID="f838e05a-0aaf-469f-aa76-40d11c33d08f" containerID="c6b53d4a9e49eaf9e7811b2e8d779de61479ec178b2430e120822932e9110415" exitCode=0 Jul 20 15:52:41.138877 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.138837 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbf99f667-bhtq5" event={"ID":"f838e05a-0aaf-469f-aa76-40d11c33d08f","Type":"ContainerDied","Data":"c6b53d4a9e49eaf9e7811b2e8d779de61479ec178b2430e120822932e9110415"} Jul 20 15:52:41.335295 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.335258 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.504609 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-config\") pod \"f838e05a-0aaf-469f-aa76-40d11c33d08f\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.504709 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-oauth-config\") pod \"f838e05a-0aaf-469f-aa76-40d11c33d08f\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.504744 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-serving-cert\") pod \"f838e05a-0aaf-469f-aa76-40d11c33d08f\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.504788 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-oauth-serving-cert\") pod \"f838e05a-0aaf-469f-aa76-40d11c33d08f\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.504830 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp5fg\" (UniqueName: \"kubernetes.io/projected/f838e05a-0aaf-469f-aa76-40d11c33d08f-kube-api-access-fp5fg\") pod \"f838e05a-0aaf-469f-aa76-40d11c33d08f\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.504867 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-service-ca\") pod \"f838e05a-0aaf-469f-aa76-40d11c33d08f\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.504933 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-trusted-ca-bundle\") pod \"f838e05a-0aaf-469f-aa76-40d11c33d08f\" (UID: \"f838e05a-0aaf-469f-aa76-40d11c33d08f\") " Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.505588 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f838e05a-0aaf-469f-aa76-40d11c33d08f" (UID: "f838e05a-0aaf-469f-aa76-40d11c33d08f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:52:41.506200 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.505915 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-config" (OuterVolumeSpecName: "console-config") pod "f838e05a-0aaf-469f-aa76-40d11c33d08f" (UID: "f838e05a-0aaf-469f-aa76-40d11c33d08f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:52:41.507155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.507126 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f838e05a-0aaf-469f-aa76-40d11c33d08f" (UID: "f838e05a-0aaf-469f-aa76-40d11c33d08f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:52:41.508331 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.508308 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-service-ca" (OuterVolumeSpecName: "service-ca") pod "f838e05a-0aaf-469f-aa76-40d11c33d08f" (UID: "f838e05a-0aaf-469f-aa76-40d11c33d08f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:52:41.515171 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.515090 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f838e05a-0aaf-469f-aa76-40d11c33d08f-kube-api-access-fp5fg" (OuterVolumeSpecName: "kube-api-access-fp5fg") pod "f838e05a-0aaf-469f-aa76-40d11c33d08f" (UID: "f838e05a-0aaf-469f-aa76-40d11c33d08f"). InnerVolumeSpecName "kube-api-access-fp5fg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:52:41.515648 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.515591 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f838e05a-0aaf-469f-aa76-40d11c33d08f" (UID: "f838e05a-0aaf-469f-aa76-40d11c33d08f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:52:41.521975 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.521932 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f838e05a-0aaf-469f-aa76-40d11c33d08f" (UID: "f838e05a-0aaf-469f-aa76-40d11c33d08f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:52:41.606053 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.606008 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp5fg\" (UniqueName: \"kubernetes.io/projected/f838e05a-0aaf-469f-aa76-40d11c33d08f-kube-api-access-fp5fg\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:41.606053 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.606056 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-service-ca\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:41.606258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.606074 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-trusted-ca-bundle\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:41.606258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.606089 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:41.606258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.606105 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-oauth-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:41.606258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.606123 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f838e05a-0aaf-469f-aa76-40d11c33d08f-console-serving-cert\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:41.606258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:41.606138 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f838e05a-0aaf-469f-aa76-40d11c33d08f-oauth-serving-cert\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:42.148337 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:42.148304 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbf99f667-bhtq5" event={"ID":"f838e05a-0aaf-469f-aa76-40d11c33d08f","Type":"ContainerDied","Data":"d6af57ee9571d1e2a84949ad0da1bb774d0dd438323345523bbdd7d925c21aba"} Jul 20 15:52:42.148525 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:42.148363 2559 scope.go:117] "RemoveContainer" containerID="c6b53d4a9e49eaf9e7811b2e8d779de61479ec178b2430e120822932e9110415" Jul 20 15:52:42.148525 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:42.148317 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbf99f667-bhtq5" Jul 20 15:52:42.170531 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:42.170488 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dbf99f667-bhtq5"] Jul 20 15:52:42.179386 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:42.177521 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dbf99f667-bhtq5"] Jul 20 15:52:42.318694 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:42.318653 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f838e05a-0aaf-469f-aa76-40d11c33d08f" path="/var/lib/kubelet/pods/f838e05a-0aaf-469f-aa76-40d11c33d08f/volumes" Jul 20 15:52:52.270118 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.270079 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8c96c586f-4jljc" podUID="f99e12e1-4ce9-4749-9a13-5984c77e10c8" containerName="console" containerID="cri-o://6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4" gracePeriod=15 Jul 20 15:52:52.556338 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.556317 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:52:52.591812 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.591779 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-service-ca\") pod \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " Jul 20 15:52:52.591972 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.591820 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-config\") pod \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " Jul 20 15:52:52.591972 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.591866 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrc4n\" (UniqueName: \"kubernetes.io/projected/f99e12e1-4ce9-4749-9a13-5984c77e10c8-kube-api-access-hrc4n\") pod \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " Jul 20 15:52:52.592083 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.592036 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-oauth-serving-cert\") pod \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " Jul 20 15:52:52.592145 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.592109 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-serving-cert\") pod \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " Jul 20 15:52:52.592198 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.592140 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-oauth-config\") pod \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\" (UID: \"f99e12e1-4ce9-4749-9a13-5984c77e10c8\") " Jul 20 15:52:52.592247 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.592218 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-service-ca" (OuterVolumeSpecName: "service-ca") pod "f99e12e1-4ce9-4749-9a13-5984c77e10c8" (UID: "f99e12e1-4ce9-4749-9a13-5984c77e10c8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:52:52.592297 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.592266 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-config" (OuterVolumeSpecName: "console-config") pod "f99e12e1-4ce9-4749-9a13-5984c77e10c8" (UID: "f99e12e1-4ce9-4749-9a13-5984c77e10c8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:52:52.592606 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.592582 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-service-ca\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:52.592722 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.592610 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:52.592722 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.592636 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f99e12e1-4ce9-4749-9a13-5984c77e10c8" (UID: "f99e12e1-4ce9-4749-9a13-5984c77e10c8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:52:52.594328 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.594307 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f99e12e1-4ce9-4749-9a13-5984c77e10c8" (UID: "f99e12e1-4ce9-4749-9a13-5984c77e10c8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:52:52.594730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.594700 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f99e12e1-4ce9-4749-9a13-5984c77e10c8-kube-api-access-hrc4n" (OuterVolumeSpecName: "kube-api-access-hrc4n") pod "f99e12e1-4ce9-4749-9a13-5984c77e10c8" (UID: "f99e12e1-4ce9-4749-9a13-5984c77e10c8"). InnerVolumeSpecName "kube-api-access-hrc4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:52:52.594909 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.594887 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f99e12e1-4ce9-4749-9a13-5984c77e10c8" (UID: "f99e12e1-4ce9-4749-9a13-5984c77e10c8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:52:52.693392 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.693371 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-serving-cert\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:52.693392 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.693391 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f99e12e1-4ce9-4749-9a13-5984c77e10c8-console-oauth-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:52.693517 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.693407 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrc4n\" (UniqueName: \"kubernetes.io/projected/f99e12e1-4ce9-4749-9a13-5984c77e10c8-kube-api-access-hrc4n\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:52.693517 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:52.693427 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f99e12e1-4ce9-4749-9a13-5984c77e10c8-oauth-serving-cert\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:52:53.202722 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.202662 2559 generic.go:356] "Generic (PLEG): container finished" podID="f99e12e1-4ce9-4749-9a13-5984c77e10c8" containerID="6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4" exitCode=0 Jul 20 15:52:53.202722 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.202708 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c96c586f-4jljc" event={"ID":"f99e12e1-4ce9-4749-9a13-5984c77e10c8","Type":"ContainerDied","Data":"6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4"} Jul 20 15:52:53.202928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.202733 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c96c586f-4jljc" Jul 20 15:52:53.202928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.202773 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c96c586f-4jljc" event={"ID":"f99e12e1-4ce9-4749-9a13-5984c77e10c8","Type":"ContainerDied","Data":"5c1b437895fec6b56e9a84d3f69e0966317fbde5d07ace68f1a502e2a54b8f2f"} Jul 20 15:52:53.202928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.202794 2559 scope.go:117] "RemoveContainer" containerID="6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4" Jul 20 15:52:53.211606 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.211592 2559 scope.go:117] "RemoveContainer" containerID="6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4" Jul 20 15:52:53.211870 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:52:53.211852 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4\": container with ID starting with 6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4 not found: ID does not exist" containerID="6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4" Jul 20 15:52:53.211948 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.211880 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4"} err="failed to get container status \"6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4\": rpc error: code = NotFound desc = could not find container \"6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4\": container with ID starting with 6c7e8c9d85429abaa1ba8c5ae87785bd53610ba38d33fb8739eb781ef1985bf4 not found: ID does not exist" Jul 20 15:52:53.227726 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.227701 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c96c586f-4jljc"] Jul 20 15:52:53.231175 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:53.231156 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8c96c586f-4jljc"] Jul 20 15:52:54.317921 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:54.317889 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f99e12e1-4ce9-4749-9a13-5984c77e10c8" path="/var/lib/kubelet/pods/f99e12e1-4ce9-4749-9a13-5984c77e10c8/volumes" Jul 20 15:52:57.059196 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:52:57.059164 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gxvpj" Jul 20 15:53:02.183101 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:02.183073 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:02.251738 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:02.251713 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:02.267429 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:02.267405 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:15.961074 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:15.961028 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:53:15.961568 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:15.961407 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="alertmanager" containerID="cri-o://8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd" gracePeriod=120 Jul 20 15:53:15.961568 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:15.961467 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy" containerID="cri-o://37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4" gracePeriod=120 Jul 20 15:53:15.961568 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:15.961528 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="config-reloader" containerID="cri-o://1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5" gracePeriod=120 Jul 20 15:53:15.961568 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:15.961503 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy-web" containerID="cri-o://8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc" gracePeriod=120 Jul 20 15:53:15.961819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:15.961481 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy-metric" containerID="cri-o://2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9" gracePeriod=120 Jul 20 15:53:15.961819 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:15.961493 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="prom-label-proxy" containerID="cri-o://52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908" gracePeriod=120 Jul 20 15:53:16.304406 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304375 2559 generic.go:356] "Generic (PLEG): container finished" podID="03980f14-10bc-4759-9f79-7b04a019f044" containerID="52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908" exitCode=0 Jul 20 15:53:16.304406 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304399 2559 generic.go:356] "Generic (PLEG): container finished" podID="03980f14-10bc-4759-9f79-7b04a019f044" containerID="2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9" exitCode=0 Jul 20 15:53:16.304406 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304407 2559 generic.go:356] "Generic (PLEG): container finished" podID="03980f14-10bc-4759-9f79-7b04a019f044" containerID="37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4" exitCode=0 Jul 20 15:53:16.304406 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304414 2559 generic.go:356] "Generic (PLEG): container finished" podID="03980f14-10bc-4759-9f79-7b04a019f044" containerID="1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5" exitCode=0 Jul 20 15:53:16.304656 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304419 2559 generic.go:356] "Generic (PLEG): container finished" podID="03980f14-10bc-4759-9f79-7b04a019f044" containerID="8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd" exitCode=0 Jul 20 15:53:16.304656 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304453 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerDied","Data":"52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908"} Jul 20 15:53:16.304656 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304524 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerDied","Data":"2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9"} Jul 20 15:53:16.304656 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304551 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerDied","Data":"37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4"} Jul 20 15:53:16.304656 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304585 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerDied","Data":"1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5"} Jul 20 15:53:16.304656 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:16.304609 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerDied","Data":"8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd"} Jul 20 15:53:17.215818 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.215793 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.309892 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.309801 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-metrics-client-ca\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.309892 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.309848 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-trusted-ca-bundle\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.309897 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-metric\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.309920 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-tls-assets\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.309946 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-config-volume\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.309974 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-main-db\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.309998 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-cluster-tls-config\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310040 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310071 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-web-config\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310103 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-web\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310504 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310144 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310504 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310192 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzqdg\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-kube-api-access-jzqdg\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310504 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310224 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-config-out\") pod \"03980f14-10bc-4759-9f79-7b04a019f044\" (UID: \"03980f14-10bc-4759-9f79-7b04a019f044\") " Jul 20 15:53:17.310504 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310324 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:17.310504 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310336 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:17.311157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310836 2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-metrics-client-ca\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.311157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310864 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-main-db\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.311157 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.310984 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:17.313863 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.313589 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:17.314244 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.314202 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:53:17.314351 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.314313 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-config-volume" (OuterVolumeSpecName: "config-volume") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:17.314995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.314545 2559 generic.go:356] "Generic (PLEG): container finished" podID="03980f14-10bc-4759-9f79-7b04a019f044" containerID="8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc" exitCode=0 Jul 20 15:53:17.314995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.314715 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerDied","Data":"8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc"} Jul 20 15:53:17.314995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.314728 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.314995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.314794 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03980f14-10bc-4759-9f79-7b04a019f044","Type":"ContainerDied","Data":"be4e5b855622e679778fe685cdff14630732ae1c11b97bc23155dc352c910c7d"} Jul 20 15:53:17.314995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.314843 2559 scope.go:117] "RemoveContainer" containerID="52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908" Jul 20 15:53:17.317361 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.317323 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-kube-api-access-jzqdg" (OuterVolumeSpecName: "kube-api-access-jzqdg") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "kube-api-access-jzqdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:53:17.317499 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.317468 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:17.317611 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.317592 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-config-out" (OuterVolumeSpecName: "config-out") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:17.318882 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.318830 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:17.319612 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.319581 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:17.321954 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.321924 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:17.331868 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.331825 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-web-config" (OuterVolumeSpecName: "web-config") pod "03980f14-10bc-4759-9f79-7b04a019f044" (UID: "03980f14-10bc-4759-9f79-7b04a019f044"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:17.338187 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.338157 2559 scope.go:117] "RemoveContainer" containerID="2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9" Jul 20 15:53:17.346455 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.346423 2559 scope.go:117] "RemoveContainer" containerID="37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4" Jul 20 15:53:17.355117 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.355087 2559 scope.go:117] "RemoveContainer" containerID="8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc" Jul 20 15:53:17.364183 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.364152 2559 scope.go:117] "RemoveContainer" containerID="1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5" Jul 20 15:53:17.372829 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.372800 2559 scope.go:117] "RemoveContainer" containerID="8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd" Jul 20 15:53:17.381536 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.381516 2559 scope.go:117] "RemoveContainer" containerID="27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8" Jul 20 15:53:17.389920 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.389895 2559 scope.go:117] "RemoveContainer" containerID="52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908" Jul 20 15:53:17.390274 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:53:17.390252 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908\": container with ID starting with 52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908 not found: ID does not exist" containerID="52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908" Jul 20 15:53:17.390322 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.390286 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908"} err="failed to get container status \"52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908\": rpc error: code = NotFound desc = could not find container \"52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908\": container with ID starting with 52a3cff87e1ad0dbce60116d5c3ed0aec38fb3f46fb51129953ed4eb1ba96908 not found: ID does not exist" Jul 20 15:53:17.390322 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.390306 2559 scope.go:117] "RemoveContainer" containerID="2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9" Jul 20 15:53:17.390589 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:53:17.390569 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9\": container with ID starting with 2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9 not found: ID does not exist" containerID="2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9" Jul 20 15:53:17.390644 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.390596 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9"} err="failed to get container status \"2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9\": rpc error: code = NotFound desc = could not find container \"2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9\": container with ID starting with 2144ac8b5f46c93f88d56c88318d735b02aeab58abeed1a3810c88d893e51cb9 not found: ID does not exist" Jul 20 15:53:17.390644 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.390612 2559 scope.go:117] "RemoveContainer" containerID="37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4" Jul 20 15:53:17.390899 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:53:17.390869 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4\": container with ID starting with 37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4 not found: ID does not exist" containerID="37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4" Jul 20 15:53:17.390899 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.390894 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4"} err="failed to get container status \"37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4\": rpc error: code = NotFound desc = could not find container \"37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4\": container with ID starting with 37b4a15db1774a40050b1a882a243c62daeb9004bb385cf42087b9268155a5f4 not found: ID does not exist" Jul 20 15:53:17.391074 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.390910 2559 scope.go:117] "RemoveContainer" containerID="8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc" Jul 20 15:53:17.391193 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:53:17.391176 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc\": container with ID starting with 8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc not found: ID does not exist" containerID="8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc" Jul 20 15:53:17.391234 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.391198 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc"} err="failed to get container status \"8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc\": rpc error: code = NotFound desc = could not find container \"8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc\": container with ID starting with 8a0d1da737c8b4ea4784f11e56f5bcae121ced99fd7b2631f426bb831ce1aacc not found: ID does not exist" Jul 20 15:53:17.391234 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.391215 2559 scope.go:117] "RemoveContainer" containerID="1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5" Jul 20 15:53:17.391469 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:53:17.391451 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5\": container with ID starting with 1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5 not found: ID does not exist" containerID="1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5" Jul 20 15:53:17.391516 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.391473 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5"} err="failed to get container status \"1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5\": rpc error: code = NotFound desc = could not find container \"1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5\": container with ID starting with 1fb2e4d2dc242c84a629ede3c62ede50b1f6a3975e3c9215c98cb3aa07906fb5 not found: ID does not exist" Jul 20 15:53:17.391516 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.391487 2559 scope.go:117] "RemoveContainer" containerID="8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd" Jul 20 15:53:17.391784 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:53:17.391768 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd\": container with ID starting with 8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd not found: ID does not exist" containerID="8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd" Jul 20 15:53:17.391834 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.391788 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd"} err="failed to get container status \"8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd\": rpc error: code = NotFound desc = could not find container \"8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd\": container with ID starting with 8a24403d3bb37e987a7169f066f47b4521b31a74db75614d9fe9865857a882fd not found: ID does not exist" Jul 20 15:53:17.391834 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.391802 2559 scope.go:117] "RemoveContainer" containerID="27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8" Jul 20 15:53:17.392044 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:53:17.392030 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8\": container with ID starting with 27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8 not found: ID does not exist" containerID="27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8" Jul 20 15:53:17.392108 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.392045 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8"} err="failed to get container status \"27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8\": rpc error: code = NotFound desc = could not find container \"27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8\": container with ID starting with 27bc52880ff990bd3cb2c740bd02de46c3d8858b2b51d715e7913cd65d620ef8 not found: ID does not exist" Jul 20 15:53:17.411549 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411500 2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-cluster-tls-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411549 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411544 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411549 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411555 2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-web-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411549 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411563 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411575 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-main-tls\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411584 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzqdg\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-kube-api-access-jzqdg\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411592 2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03980f14-10bc-4759-9f79-7b04a019f044-config-out\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411600 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03980f14-10bc-4759-9f79-7b04a019f044-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411610 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411618 2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03980f14-10bc-4759-9f79-7b04a019f044-tls-assets\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.411788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.411627 2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03980f14-10bc-4759-9f79-7b04a019f044-config-volume\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:17.654736 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.654700 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:53:17.660538 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.660503 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:53:17.687985 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.687950 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:53:17.688270 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688257 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy-web" Jul 20 15:53:17.688313 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688271 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy-web" Jul 20 15:53:17.688313 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688285 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy" Jul 20 15:53:17.688313 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688290 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy" Jul 20 15:53:17.688313 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688297 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="prom-label-proxy" Jul 20 15:53:17.688313 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688302 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="prom-label-proxy" Jul 20 15:53:17.688313 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688310 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="config-reloader" Jul 20 15:53:17.688313 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688315 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="config-reloader" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688324 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f99e12e1-4ce9-4749-9a13-5984c77e10c8" containerName="console" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688329 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f99e12e1-4ce9-4749-9a13-5984c77e10c8" containerName="console" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688340 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="init-config-reloader" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688345 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="init-config-reloader" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688352 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy-metric" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688357 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy-metric" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688363 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="alertmanager" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688368 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="alertmanager" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688375 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f838e05a-0aaf-469f-aa76-40d11c33d08f" containerName="console" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688379 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f838e05a-0aaf-469f-aa76-40d11c33d08f" containerName="console" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688428 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="prom-label-proxy" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688437 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="config-reloader" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688443 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f99e12e1-4ce9-4749-9a13-5984c77e10c8" containerName="console" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688449 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="alertmanager" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688456 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy-metric" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688464 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f838e05a-0aaf-469f-aa76-40d11c33d08f" containerName="console" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688473 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy-web" Jul 20 15:53:17.688508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.688479 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03980f14-10bc-4759-9f79-7b04a019f044" containerName="kube-rbac-proxy" Jul 20 15:53:17.698426 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.698384 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.707734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.707706 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 15:53:17.707907 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.707706 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 15:53:17.707907 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.707860 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4lntm\"" Jul 20 15:53:17.708016 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.707934 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 15:53:17.708016 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.707975 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 15:53:17.708255 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.708240 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 15:53:17.708329 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.708262 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 15:53:17.708329 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.708277 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 15:53:17.708426 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.708241 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 15:53:17.715156 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.715130 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:53:17.720662 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.720640 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 15:53:17.814402 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814352 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814402 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814406 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/830ea423-114e-4156-97c7-83aa8ba58190-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814429 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814447 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-config-volume\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814519 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814557 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814576 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814593 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l98s\" (UniqueName: \"kubernetes.io/projected/830ea423-114e-4156-97c7-83aa8ba58190-kube-api-access-6l98s\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814617 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/830ea423-114e-4156-97c7-83aa8ba58190-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814880 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814666 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-web-config\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814880 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814710 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/830ea423-114e-4156-97c7-83aa8ba58190-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814880 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/830ea423-114e-4156-97c7-83aa8ba58190-tls-assets\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.814880 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.814764 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/830ea423-114e-4156-97c7-83aa8ba58190-config-out\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916103 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916010 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916103 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916052 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916103 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916071 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916103 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916091 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l98s\" (UniqueName: \"kubernetes.io/projected/830ea423-114e-4156-97c7-83aa8ba58190-kube-api-access-6l98s\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/830ea423-114e-4156-97c7-83aa8ba58190-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-web-config\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916274 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/830ea423-114e-4156-97c7-83aa8ba58190-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/830ea423-114e-4156-97c7-83aa8ba58190-tls-assets\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916413 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916392 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/830ea423-114e-4156-97c7-83aa8ba58190-config-out\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916659 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916659 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/830ea423-114e-4156-97c7-83aa8ba58190-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916659 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916500 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916659 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916537 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/830ea423-114e-4156-97c7-83aa8ba58190-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.916659 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916537 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-config-volume\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.917097 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.916994 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/830ea423-114e-4156-97c7-83aa8ba58190-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.918003 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.917976 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/830ea423-114e-4156-97c7-83aa8ba58190-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.919379 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.919351 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/830ea423-114e-4156-97c7-83aa8ba58190-tls-assets\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.919939 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.919503 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-web-config\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.919939 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.919641 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.919939 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.919734 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-config-volume\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.919939 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.919858 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.919939 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.919880 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.919939 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.919897 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.920631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.920610 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/830ea423-114e-4156-97c7-83aa8ba58190-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.921287 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.921269 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/830ea423-114e-4156-97c7-83aa8ba58190-config-out\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:17.932406 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:17.932380 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l98s\" (UniqueName: \"kubernetes.io/projected/830ea423-114e-4156-97c7-83aa8ba58190-kube-api-access-6l98s\") pod \"alertmanager-main-0\" (UID: \"830ea423-114e-4156-97c7-83aa8ba58190\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:18.007775 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:18.007740 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:53:18.184123 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:53:18.184093 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830ea423_114e_4156_97c7_83aa8ba58190.slice/crio-2b831ca41825c8999da5f7f5e15d8d3dfbc3f4335133738c85dee5dcf36a2944 WatchSource:0}: Error finding container 2b831ca41825c8999da5f7f5e15d8d3dfbc3f4335133738c85dee5dcf36a2944: Status 404 returned error can't find the container with id 2b831ca41825c8999da5f7f5e15d8d3dfbc3f4335133738c85dee5dcf36a2944 Jul 20 15:53:18.185831 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:18.185802 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:53:18.318537 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:18.318497 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03980f14-10bc-4759-9f79-7b04a019f044" path="/var/lib/kubelet/pods/03980f14-10bc-4759-9f79-7b04a019f044/volumes" Jul 20 15:53:18.320133 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:18.320100 2559 generic.go:356] "Generic (PLEG): container finished" podID="830ea423-114e-4156-97c7-83aa8ba58190" containerID="17099363af322208b461b6ac4ddb74592340931cd5168048732316b4e815bb1f" exitCode=0 Jul 20 15:53:18.320293 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:18.320180 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"830ea423-114e-4156-97c7-83aa8ba58190","Type":"ContainerDied","Data":"17099363af322208b461b6ac4ddb74592340931cd5168048732316b4e815bb1f"} Jul 20 15:53:18.320293 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:18.320247 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"830ea423-114e-4156-97c7-83aa8ba58190","Type":"ContainerStarted","Data":"2b831ca41825c8999da5f7f5e15d8d3dfbc3f4335133738c85dee5dcf36a2944"} Jul 20 15:53:19.329806 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:19.329704 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"830ea423-114e-4156-97c7-83aa8ba58190","Type":"ContainerStarted","Data":"b85e1f4f314af99002f3d8a36dfb1010d4d281c8a58376cfc499af95c9b493f8"} Jul 20 15:53:19.329806 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:19.329774 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"830ea423-114e-4156-97c7-83aa8ba58190","Type":"ContainerStarted","Data":"606278219c905017ec0249ec7c459182523c0a3c6f2378db7e5754e3e9a02c7e"} Jul 20 15:53:19.329806 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:19.329800 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"830ea423-114e-4156-97c7-83aa8ba58190","Type":"ContainerStarted","Data":"682c9ef864e1dd9a987612c1b55f02eb364d7f03df30ab82e53f16c162010c1c"} Jul 20 15:53:19.330262 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:19.329825 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"830ea423-114e-4156-97c7-83aa8ba58190","Type":"ContainerStarted","Data":"a3453d15fc38b7b43ae3496aac44a6275d87fac4e25dbbc386de80d7966c4ebb"} Jul 20 15:53:19.330262 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:19.329855 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"830ea423-114e-4156-97c7-83aa8ba58190","Type":"ContainerStarted","Data":"4646810c8ac74df9549cd58d447e4cd692fd9e7552aa933a131388f544f4fefe"} Jul 20 15:53:19.330262 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:19.329880 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"830ea423-114e-4156-97c7-83aa8ba58190","Type":"ContainerStarted","Data":"1a43623b136baa379e58b7ceec907d51c13540c9482b850a2142d01978dffb49"} Jul 20 15:53:19.374892 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:19.374814 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.359382253 podStartE2EDuration="2.374786879s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.404626ms" isStatefulPod=false podCreationTimestamp="2026-07-20 15:53:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:53:19.370553746 +0000 UTC m=+137.549230819" watchObservedRunningTime="2026-07-20 15:53:19.374786879 +0000 UTC m=+137.553463953" Jul 20 15:53:20.205062 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.205025 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:53:20.205812 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.205396 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="prometheus" containerID="cri-o://360c126beb5410f524dbbe33b0d486556096284977065e344a06535c9b97bb03" gracePeriod=600 Jul 20 15:53:20.205812 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.205452 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy-web" containerID="cri-o://0961be1f3e4832ed77dc8641799addc78af686b5c11f155d9e41f96d27c0ebb3" gracePeriod=600 Jul 20 15:53:20.205812 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.205488 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="config-reloader" containerID="cri-o://95b853f8259c101d51e231259645cdc2ffb1874b26ee0aadf984745ec5aa6af2" gracePeriod=600 Jul 20 15:53:20.205812 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.205521 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://34ca26cd604b17f78eea2026cb4065d7f44b2d3e7ee0dd16d940c36e04b2d349" gracePeriod=600 Jul 20 15:53:20.205812 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.205458 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="thanos-sidecar" containerID="cri-o://f6f164657b70f740290b989d24f7962b69cbb5b05272265e5c3008e9441b1be4" gracePeriod=600 Jul 20 15:53:20.205812 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.205604 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy" containerID="cri-o://1d8aeadb7533371e8c37789e1a32f0b7b3d726dcb612de821e84019ee36623b4" gracePeriod=600 Jul 20 15:53:20.342038 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.341120 2559 generic.go:356] "Generic (PLEG): container finished" podID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerID="34ca26cd604b17f78eea2026cb4065d7f44b2d3e7ee0dd16d940c36e04b2d349" exitCode=0 Jul 20 15:53:20.342038 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.341163 2559 generic.go:356] "Generic (PLEG): container finished" podID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerID="1d8aeadb7533371e8c37789e1a32f0b7b3d726dcb612de821e84019ee36623b4" exitCode=0 Jul 20 15:53:20.342038 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.341177 2559 generic.go:356] "Generic (PLEG): container finished" podID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerID="0961be1f3e4832ed77dc8641799addc78af686b5c11f155d9e41f96d27c0ebb3" exitCode=0 Jul 20 15:53:20.342038 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.341190 2559 generic.go:356] "Generic (PLEG): container finished" podID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerID="f6f164657b70f740290b989d24f7962b69cbb5b05272265e5c3008e9441b1be4" exitCode=0 Jul 20 15:53:20.342038 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.341203 2559 generic.go:356] "Generic (PLEG): container finished" podID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerID="95b853f8259c101d51e231259645cdc2ffb1874b26ee0aadf984745ec5aa6af2" exitCode=0 Jul 20 15:53:20.342038 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.341215 2559 generic.go:356] "Generic (PLEG): container finished" podID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerID="360c126beb5410f524dbbe33b0d486556096284977065e344a06535c9b97bb03" exitCode=0 Jul 20 15:53:20.342752 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.342078 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerDied","Data":"34ca26cd604b17f78eea2026cb4065d7f44b2d3e7ee0dd16d940c36e04b2d349"} Jul 20 15:53:20.342752 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.342163 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerDied","Data":"1d8aeadb7533371e8c37789e1a32f0b7b3d726dcb612de821e84019ee36623b4"} Jul 20 15:53:20.342752 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.342206 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerDied","Data":"0961be1f3e4832ed77dc8641799addc78af686b5c11f155d9e41f96d27c0ebb3"} Jul 20 15:53:20.342752 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.342247 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerDied","Data":"f6f164657b70f740290b989d24f7962b69cbb5b05272265e5c3008e9441b1be4"} Jul 20 15:53:20.342752 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.342285 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerDied","Data":"95b853f8259c101d51e231259645cdc2ffb1874b26ee0aadf984745ec5aa6af2"} Jul 20 15:53:20.342752 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.342324 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerDied","Data":"360c126beb5410f524dbbe33b0d486556096284977065e344a06535c9b97bb03"} Jul 20 15:53:20.469482 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.469453 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:20.541602 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.541568 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-kubelet-serving-ca-bundle\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.541602 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.541603 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ndnm\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-kube-api-access-7ndnm\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.541869 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.541637 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-web-config\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.541869 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.541837 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-db\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.541989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.541923 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-config\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.541989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.541954 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-tls-assets\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.541989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.541952 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:20.541989 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.541981 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-thanos-prometheus-http-client-file\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.542183 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542031 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-0\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.542183 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542066 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-serving-certs-ca-bundle\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.542183 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542102 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-trusted-ca-bundle\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.542183 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542131 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-config-out\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.542183 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542163 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-2\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.542421 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542195 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-kube-rbac-proxy\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.542421 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542244 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542627 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542730 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542739 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-metrics-client-certs\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542786 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-metrics-client-ca\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542811 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-grpc-tls\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542843 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542874 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-1\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542890 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:20.543190 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.542899 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-tls\") pod \"81b9a5ce-98af-4a95-a982-182982fc1ee8\" (UID: \"81b9a5ce-98af-4a95-a982-182982fc1ee8\") " Jul 20 15:53:20.543723 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.543223 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.543723 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.543243 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-db\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.543723 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.543257 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.543723 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.543274 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.543723 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.543377 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:20.544848 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.544819 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:20.545406 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.545379 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:20.545999 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.545946 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:20.548176 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.547796 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.548176 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.547886 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-config-out" (OuterVolumeSpecName: "config-out") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:20.548176 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.548150 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.548421 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.548283 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-kube-api-access-7ndnm" (OuterVolumeSpecName: "kube-api-access-7ndnm") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "kube-api-access-7ndnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:53:20.548421 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.548306 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.548421 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.548415 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-config" (OuterVolumeSpecName: "config") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.548584 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.548450 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:53:20.548923 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.548900 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.549022 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.548998 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.549127 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.549074 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.549165 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.549138 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.562925 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.562895 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-web-config" (OuterVolumeSpecName: "web-config") pod "81b9a5ce-98af-4a95-a982-182982fc1ee8" (UID: "81b9a5ce-98af-4a95-a982-182982fc1ee8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:53:20.643881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643843 2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-web-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.643881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643876 2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.643881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643884 2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-tls-assets\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.643881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643894 2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643903 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643912 2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81b9a5ce-98af-4a95-a982-182982fc1ee8-config-out\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643921 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643931 2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-kube-rbac-proxy\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643939 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643948 2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-metrics-client-certs\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643956 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-configmap-metrics-client-ca\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643964 2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-grpc-tls\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643972 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643983 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/81b9a5ce-98af-4a95-a982-182982fc1ee8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643991 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/81b9a5ce-98af-4a95-a982-182982fc1ee8-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:20.644155 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:20.643999 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ndnm\" (UniqueName: \"kubernetes.io/projected/81b9a5ce-98af-4a95-a982-182982fc1ee8-kube-api-access-7ndnm\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:53:21.348632 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.348592 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"81b9a5ce-98af-4a95-a982-182982fc1ee8","Type":"ContainerDied","Data":"b1846d70a3ccd2bae5d38109c0a6331e23c51587ff225ba33e2234f4396054ac"} Jul 20 15:53:21.348632 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.348631 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.349099 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.348670 2559 scope.go:117] "RemoveContainer" containerID="34ca26cd604b17f78eea2026cb4065d7f44b2d3e7ee0dd16d940c36e04b2d349" Jul 20 15:53:21.357877 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.357859 2559 scope.go:117] "RemoveContainer" containerID="1d8aeadb7533371e8c37789e1a32f0b7b3d726dcb612de821e84019ee36623b4" Jul 20 15:53:21.366201 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.366170 2559 scope.go:117] "RemoveContainer" containerID="0961be1f3e4832ed77dc8641799addc78af686b5c11f155d9e41f96d27c0ebb3" Jul 20 15:53:21.374111 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.374082 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:53:21.375173 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.375156 2559 scope.go:117] "RemoveContainer" containerID="f6f164657b70f740290b989d24f7962b69cbb5b05272265e5c3008e9441b1be4" Jul 20 15:53:21.380211 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.380186 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:53:21.383583 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.383562 2559 scope.go:117] "RemoveContainer" containerID="95b853f8259c101d51e231259645cdc2ffb1874b26ee0aadf984745ec5aa6af2" Jul 20 15:53:21.391648 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.391625 2559 scope.go:117] "RemoveContainer" containerID="360c126beb5410f524dbbe33b0d486556096284977065e344a06535c9b97bb03" Jul 20 15:53:21.400593 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.400573 2559 scope.go:117] "RemoveContainer" containerID="bc369aa64f7c5a24ec2e89c349accf4911513f9ef041b6ecfa778ab2f64b0526" Jul 20 15:53:21.408614 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408575 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:53:21.408949 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408936 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="init-config-reloader" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408951 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="init-config-reloader" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408963 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy-thanos" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408969 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy-thanos" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408977 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="thanos-sidecar" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408983 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="thanos-sidecar" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408993 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="prometheus" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.408998 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="prometheus" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409003 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="config-reloader" Jul 20 15:53:21.409007 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409008 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="config-reloader" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409016 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy-web" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409021 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy-web" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409027 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409032 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409082 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="thanos-sidecar" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409092 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409099 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy-web" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409105 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="prometheus" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409111 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="config-reloader" Jul 20 15:53:21.409299 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.409116 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" containerName="kube-rbac-proxy-thanos" Jul 20 15:53:21.412952 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.412924 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.415129 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415102 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 20 15:53:21.415278 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415133 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 20 15:53:21.415278 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415224 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 20 15:53:21.415278 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415230 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 20 15:53:21.415278 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415102 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bwspc\"" Jul 20 15:53:21.415278 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415249 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 20 15:53:21.415595 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415577 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 20 15:53:21.415837 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415820 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 20 15:53:21.415930 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415842 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 20 15:53:21.415930 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415886 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 20 15:53:21.415930 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415906 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 20 15:53:21.416072 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415937 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dq57aocrjctde\"" Jul 20 15:53:21.416072 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.415998 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 20 15:53:21.416739 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.416719 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 20 15:53:21.423001 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.419643 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 20 15:53:21.424968 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.424934 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 20 15:53:21.430615 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.430585 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:53:21.552017 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.551986 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9aff44ab-d6d2-4093-80af-58960f9647ab-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552211 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552211 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552050 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552211 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552083 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552211 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9aff44ab-d6d2-4093-80af-58960f9647ab-config-out\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552211 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552174 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552211 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552203 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552227 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552244 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552261 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552283 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-web-config\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552307 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552359 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552390 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552406 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552422 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552445 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552837 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552544 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5zgz\" (UniqueName: \"kubernetes.io/projected/9aff44ab-d6d2-4093-80af-58960f9647ab-kube-api-access-t5zgz\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552837 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552615 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-config\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.552837 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.552632 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653453 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653403 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653453 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653460 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-web-config\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653482 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653504 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653526 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653541 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653558 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653584 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653610 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5zgz\" (UniqueName: \"kubernetes.io/projected/9aff44ab-d6d2-4093-80af-58960f9647ab-kube-api-access-t5zgz\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653642 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-config\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653660 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653714 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9aff44ab-d6d2-4093-80af-58960f9647ab-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653737 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.653765 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653764 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.654320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653808 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.654320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653850 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9aff44ab-d6d2-4093-80af-58960f9647ab-config-out\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.654320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.654320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653909 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.654320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653936 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.654320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.653962 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.654320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.654029 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.654773 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.654517 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.655154 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.655122 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.655331 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.655306 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.656955 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.656924 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.657308 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.657280 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.657631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.657609 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.657855 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.657832 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9aff44ab-d6d2-4093-80af-58960f9647ab-config-out\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.658162 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.658118 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.658533 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.658320 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.658533 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.658480 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.658764 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.658720 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-config\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.658853 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.658760 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.659053 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.659035 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-web-config\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.659139 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.659050 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9aff44ab-d6d2-4093-80af-58960f9647ab-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.659783 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.659754 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.660377 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.660354 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.660465 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.660364 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9aff44ab-d6d2-4093-80af-58960f9647ab-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.661319 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.661299 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9aff44ab-d6d2-4093-80af-58960f9647ab-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.663380 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.663360 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5zgz\" (UniqueName: \"kubernetes.io/projected/9aff44ab-d6d2-4093-80af-58960f9647ab-kube-api-access-t5zgz\") pod \"prometheus-k8s-0\" (UID: \"9aff44ab-d6d2-4093-80af-58960f9647ab\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.726486 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.726447 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:21.883807 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:21.883772 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 15:53:21.887368 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:53:21.887336 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aff44ab_d6d2_4093_80af_58960f9647ab.slice/crio-ac6dd07951cf72ab21e7399a8a1511cdfad08b0ffaa145eb77b7cd08adda7d14 WatchSource:0}: Error finding container ac6dd07951cf72ab21e7399a8a1511cdfad08b0ffaa145eb77b7cd08adda7d14: Status 404 returned error can't find the container with id ac6dd07951cf72ab21e7399a8a1511cdfad08b0ffaa145eb77b7cd08adda7d14 Jul 20 15:53:22.317591 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:22.317553 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b9a5ce-98af-4a95-a982-182982fc1ee8" path="/var/lib/kubelet/pods/81b9a5ce-98af-4a95-a982-182982fc1ee8/volumes" Jul 20 15:53:22.354316 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:22.354279 2559 generic.go:356] "Generic (PLEG): container finished" podID="9aff44ab-d6d2-4093-80af-58960f9647ab" containerID="713777f7988ac0f9edd1f14bd74e73ff6dba6ef2634c872eb14aa4b6b7bc27cb" exitCode=0 Jul 20 15:53:22.354779 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:22.354378 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9aff44ab-d6d2-4093-80af-58960f9647ab","Type":"ContainerDied","Data":"713777f7988ac0f9edd1f14bd74e73ff6dba6ef2634c872eb14aa4b6b7bc27cb"} Jul 20 15:53:22.354779 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:22.354472 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9aff44ab-d6d2-4093-80af-58960f9647ab","Type":"ContainerStarted","Data":"ac6dd07951cf72ab21e7399a8a1511cdfad08b0ffaa145eb77b7cd08adda7d14"} Jul 20 15:53:23.363289 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:23.363244 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9aff44ab-d6d2-4093-80af-58960f9647ab","Type":"ContainerStarted","Data":"a4d4c19a26bc78978b6d6a9f04616b52e93a87a0f54bd0d85d124b3559437639"} Jul 20 15:53:23.363709 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:23.363344 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9aff44ab-d6d2-4093-80af-58960f9647ab","Type":"ContainerStarted","Data":"b8ae44f4811cf210f84bb337d31e42fcf29a78aa03449765288c3aa26de58d99"} Jul 20 15:53:23.363709 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:23.363373 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9aff44ab-d6d2-4093-80af-58960f9647ab","Type":"ContainerStarted","Data":"4b3e468f4fc831c01fb404c9e3316f0e70667273327855da318276287b242e47"} Jul 20 15:53:23.363709 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:23.363399 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9aff44ab-d6d2-4093-80af-58960f9647ab","Type":"ContainerStarted","Data":"5550dba10f12fcfbccde39c5ea051ae90bab7bcc07b37eb78fd7ea1aca701750"} Jul 20 15:53:23.363709 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:23.363422 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9aff44ab-d6d2-4093-80af-58960f9647ab","Type":"ContainerStarted","Data":"89985003fdf27579569f407c459e3ae14f7340c64e90f096d7fa025bd148ddc6"} Jul 20 15:53:23.363709 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:23.363449 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9aff44ab-d6d2-4093-80af-58960f9647ab","Type":"ContainerStarted","Data":"07e0b939c8f1f16441161bb6f52019bc72be153e15c38b92ce0bbef77e65ebea"} Jul 20 15:53:23.414725 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:23.414644 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.393808288 podStartE2EDuration="2.414626238s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.81795ms" isStatefulPod=false podCreationTimestamp="2026-07-20 15:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:53:23.408821735 +0000 UTC m=+141.587498808" watchObservedRunningTime="2026-07-20 15:53:23.414626238 +0000 UTC m=+141.593303310" Jul 20 15:53:26.726657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:26.726621 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:53:44.967561 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:44.967529 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6df75c8764-2rzhf"] Jul 20 15:53:44.971841 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:44.971818 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:44.981657 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:44.981627 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6df75c8764-2rzhf"] Jul 20 15:53:45.066666 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.066630 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-console-config\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.066666 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.066667 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-oauth-serving-cert\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.066972 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.066766 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-trusted-ca-bundle\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.066972 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.066791 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-oauth-config\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.066972 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.066837 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxk2g\" (UniqueName: \"kubernetes.io/projected/c26b38cd-d4ff-43da-8c17-33487944db68-kube-api-access-dxk2g\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.066972 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.066858 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-service-ca\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.066972 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.066932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-serving-cert\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.168368 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.168331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-console-config\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.168368 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.168366 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-oauth-serving-cert\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.168594 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.168399 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-trusted-ca-bundle\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.168594 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.168418 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-oauth-config\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.168594 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.168445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxk2g\" (UniqueName: \"kubernetes.io/projected/c26b38cd-d4ff-43da-8c17-33487944db68-kube-api-access-dxk2g\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.168755 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.168576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-service-ca\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.168755 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.168636 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-serving-cert\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.169249 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.169222 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-oauth-serving-cert\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.169249 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.169240 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-service-ca\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.169456 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.169417 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-trusted-ca-bundle\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.169554 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.169465 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-console-config\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.171038 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.171021 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-oauth-config\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.171230 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.171208 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-serving-cert\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.176840 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.176816 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxk2g\" (UniqueName: \"kubernetes.io/projected/c26b38cd-d4ff-43da-8c17-33487944db68-kube-api-access-dxk2g\") pod \"console-6df75c8764-2rzhf\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.281899 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.281807 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:45.422254 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.422217 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6df75c8764-2rzhf"] Jul 20 15:53:45.425469 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:53:45.425430 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26b38cd_d4ff_43da_8c17_33487944db68.slice/crio-363cf8e00e5716c6705ad87350b474d408ff33573d321218dd84551da5c7ac6d WatchSource:0}: Error finding container 363cf8e00e5716c6705ad87350b474d408ff33573d321218dd84551da5c7ac6d: Status 404 returned error can't find the container with id 363cf8e00e5716c6705ad87350b474d408ff33573d321218dd84551da5c7ac6d Jul 20 15:53:45.457416 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:45.457383 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6df75c8764-2rzhf" event={"ID":"c26b38cd-d4ff-43da-8c17-33487944db68","Type":"ContainerStarted","Data":"363cf8e00e5716c6705ad87350b474d408ff33573d321218dd84551da5c7ac6d"} Jul 20 15:53:46.462947 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:46.462901 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6df75c8764-2rzhf" event={"ID":"c26b38cd-d4ff-43da-8c17-33487944db68","Type":"ContainerStarted","Data":"b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56"} Jul 20 15:53:46.481983 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:46.481926 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6df75c8764-2rzhf" podStartSLOduration=2.481909034 podStartE2EDuration="2.481909034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:53:46.480237516 +0000 UTC m=+164.658914590" watchObservedRunningTime="2026-07-20 15:53:46.481909034 +0000 UTC m=+164.660586118" Jul 20 15:53:55.282419 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:55.282328 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:55.282419 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:55.282386 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:55.287456 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:55.287426 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:55.505934 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:55.505904 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:53:55.556029 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:53:55.555934 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-584dcbdb6b-hxgmc"] Jul 20 15:54:20.582816 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.582772 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-584dcbdb6b-hxgmc" podUID="921105af-900c-451b-9446-01ad3d088ccb" containerName="console" containerID="cri-o://ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7" gracePeriod=15 Jul 20 15:54:20.828843 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.828819 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:54:20.903804 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.903709 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-oauth-serving-cert\") pod \"921105af-900c-451b-9446-01ad3d088ccb\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " Jul 20 15:54:20.903804 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.903766 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-service-ca\") pod \"921105af-900c-451b-9446-01ad3d088ccb\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " Jul 20 15:54:20.904009 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.903820 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-trusted-ca-bundle\") pod \"921105af-900c-451b-9446-01ad3d088ccb\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " Jul 20 15:54:20.904009 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.903849 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-console-config\") pod \"921105af-900c-451b-9446-01ad3d088ccb\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " Jul 20 15:54:20.904009 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.903892 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctp7h\" (UniqueName: \"kubernetes.io/projected/921105af-900c-451b-9446-01ad3d088ccb-kube-api-access-ctp7h\") pod \"921105af-900c-451b-9446-01ad3d088ccb\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " Jul 20 15:54:20.904009 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.903924 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-oauth-config\") pod \"921105af-900c-451b-9446-01ad3d088ccb\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " Jul 20 15:54:20.904009 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.903985 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-serving-cert\") pod \"921105af-900c-451b-9446-01ad3d088ccb\" (UID: \"921105af-900c-451b-9446-01ad3d088ccb\") " Jul 20 15:54:20.904250 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.904145 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "921105af-900c-451b-9446-01ad3d088ccb" (UID: "921105af-900c-451b-9446-01ad3d088ccb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:54:20.904330 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.904312 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-oauth-serving-cert\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:54:20.904389 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.904343 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "921105af-900c-451b-9446-01ad3d088ccb" (UID: "921105af-900c-451b-9446-01ad3d088ccb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:54:20.904389 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.904351 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-service-ca" (OuterVolumeSpecName: "service-ca") pod "921105af-900c-451b-9446-01ad3d088ccb" (UID: "921105af-900c-451b-9446-01ad3d088ccb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:54:20.904505 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.904385 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-console-config" (OuterVolumeSpecName: "console-config") pod "921105af-900c-451b-9446-01ad3d088ccb" (UID: "921105af-900c-451b-9446-01ad3d088ccb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:54:20.906447 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.906411 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/921105af-900c-451b-9446-01ad3d088ccb-kube-api-access-ctp7h" (OuterVolumeSpecName: "kube-api-access-ctp7h") pod "921105af-900c-451b-9446-01ad3d088ccb" (UID: "921105af-900c-451b-9446-01ad3d088ccb"). InnerVolumeSpecName "kube-api-access-ctp7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:54:20.906447 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.906419 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "921105af-900c-451b-9446-01ad3d088ccb" (UID: "921105af-900c-451b-9446-01ad3d088ccb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:54:20.906600 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:20.906459 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "921105af-900c-451b-9446-01ad3d088ccb" (UID: "921105af-900c-451b-9446-01ad3d088ccb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:54:21.004988 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.004953 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-console-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:54:21.004988 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.004983 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctp7h\" (UniqueName: \"kubernetes.io/projected/921105af-900c-451b-9446-01ad3d088ccb-kube-api-access-ctp7h\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:54:21.004988 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.004992 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-oauth-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:54:21.005219 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.005001 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/921105af-900c-451b-9446-01ad3d088ccb-console-serving-cert\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:54:21.005219 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.005010 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-service-ca\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:54:21.005219 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.005020 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921105af-900c-451b-9446-01ad3d088ccb-trusted-ca-bundle\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:54:21.612974 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.612937 2559 generic.go:356] "Generic (PLEG): container finished" podID="921105af-900c-451b-9446-01ad3d088ccb" containerID="ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7" exitCode=0 Jul 20 15:54:21.613388 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.613000 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584dcbdb6b-hxgmc" Jul 20 15:54:21.613388 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.613026 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584dcbdb6b-hxgmc" event={"ID":"921105af-900c-451b-9446-01ad3d088ccb","Type":"ContainerDied","Data":"ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7"} Jul 20 15:54:21.613388 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.613081 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584dcbdb6b-hxgmc" event={"ID":"921105af-900c-451b-9446-01ad3d088ccb","Type":"ContainerDied","Data":"32bb85d932219878839b27fb55941a4689401b0c69fc7fec9c7564a7e53ad94c"} Jul 20 15:54:21.613388 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.613102 2559 scope.go:117] "RemoveContainer" containerID="ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7" Jul 20 15:54:21.623987 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.623966 2559 scope.go:117] "RemoveContainer" containerID="ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7" Jul 20 15:54:21.624372 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:54:21.624348 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7\": container with ID starting with ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7 not found: ID does not exist" containerID="ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7" Jul 20 15:54:21.624456 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.624381 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7"} err="failed to get container status \"ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7\": rpc error: code = NotFound desc = could not find container \"ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7\": container with ID starting with ada822805439d3105c0eca0b83ce16c028a69ab1aacf66fbc0a213bf0dae1cc7 not found: ID does not exist" Jul 20 15:54:21.635481 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.635453 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-584dcbdb6b-hxgmc"] Jul 20 15:54:21.639372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.639344 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-584dcbdb6b-hxgmc"] Jul 20 15:54:21.726988 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.726952 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:54:21.743449 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:21.743426 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:54:22.317143 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:22.317112 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="921105af-900c-451b-9446-01ad3d088ccb" path="/var/lib/kubelet/pods/921105af-900c-451b-9446-01ad3d088ccb/volumes" Jul 20 15:54:22.635162 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:22.635084 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 15:54:52.218781 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.218750 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-gpxqn"] Jul 20 15:54:52.219258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.219091 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="921105af-900c-451b-9446-01ad3d088ccb" containerName="console" Jul 20 15:54:52.219258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.219105 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="921105af-900c-451b-9446-01ad3d088ccb" containerName="console" Jul 20 15:54:52.219258 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.219156 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="921105af-900c-451b-9446-01ad3d088ccb" containerName="console" Jul 20 15:54:52.221872 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.221857 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-gpxqn" Jul 20 15:54:52.224050 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.224030 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 15:54:52.224555 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.224539 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qq6wh\"" Jul 20 15:54:52.224631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.224566 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 15:54:52.233594 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.233568 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-gpxqn"] Jul 20 15:54:52.405193 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.405161 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6bd290a-ada8-40a6-b135-727cea6d4768-bound-sa-token\") pod \"cert-manager-55b4d954d5-gpxqn\" (UID: \"b6bd290a-ada8-40a6-b135-727cea6d4768\") " pod="cert-manager/cert-manager-55b4d954d5-gpxqn" Jul 20 15:54:52.405362 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.405202 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfm45\" (UniqueName: \"kubernetes.io/projected/b6bd290a-ada8-40a6-b135-727cea6d4768-kube-api-access-lfm45\") pod \"cert-manager-55b4d954d5-gpxqn\" (UID: \"b6bd290a-ada8-40a6-b135-727cea6d4768\") " pod="cert-manager/cert-manager-55b4d954d5-gpxqn" Jul 20 15:54:52.506759 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.506665 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6bd290a-ada8-40a6-b135-727cea6d4768-bound-sa-token\") pod \"cert-manager-55b4d954d5-gpxqn\" (UID: \"b6bd290a-ada8-40a6-b135-727cea6d4768\") " pod="cert-manager/cert-manager-55b4d954d5-gpxqn" Jul 20 15:54:52.506759 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.506728 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfm45\" (UniqueName: \"kubernetes.io/projected/b6bd290a-ada8-40a6-b135-727cea6d4768-kube-api-access-lfm45\") pod \"cert-manager-55b4d954d5-gpxqn\" (UID: \"b6bd290a-ada8-40a6-b135-727cea6d4768\") " pod="cert-manager/cert-manager-55b4d954d5-gpxqn" Jul 20 15:54:52.515885 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.515851 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6bd290a-ada8-40a6-b135-727cea6d4768-bound-sa-token\") pod \"cert-manager-55b4d954d5-gpxqn\" (UID: \"b6bd290a-ada8-40a6-b135-727cea6d4768\") " pod="cert-manager/cert-manager-55b4d954d5-gpxqn" Jul 20 15:54:52.516014 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.515917 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfm45\" (UniqueName: \"kubernetes.io/projected/b6bd290a-ada8-40a6-b135-727cea6d4768-kube-api-access-lfm45\") pod \"cert-manager-55b4d954d5-gpxqn\" (UID: \"b6bd290a-ada8-40a6-b135-727cea6d4768\") " pod="cert-manager/cert-manager-55b4d954d5-gpxqn" Jul 20 15:54:52.548225 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.548191 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-gpxqn" Jul 20 15:54:52.687294 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.687239 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-gpxqn"] Jul 20 15:54:52.692373 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:54:52.692340 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6bd290a_ada8_40a6_b135_727cea6d4768.slice/crio-8326d65c54d0eced369564db2ee547269313b58ec2daa3c446558780cef1b843 WatchSource:0}: Error finding container 8326d65c54d0eced369564db2ee547269313b58ec2daa3c446558780cef1b843: Status 404 returned error can't find the container with id 8326d65c54d0eced369564db2ee547269313b58ec2daa3c446558780cef1b843 Jul 20 15:54:52.742283 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:52.742245 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-gpxqn" event={"ID":"b6bd290a-ada8-40a6-b135-727cea6d4768","Type":"ContainerStarted","Data":"8326d65c54d0eced369564db2ee547269313b58ec2daa3c446558780cef1b843"} Jul 20 15:54:55.468422 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.468386 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9"] Jul 20 15:54:55.472149 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.472132 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" Jul 20 15:54:55.474868 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.474843 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 15:54:55.475127 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.475111 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-c6dwj\"" Jul 20 15:54:55.475467 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.475441 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 15:54:55.484188 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.484159 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9"] Jul 20 15:54:55.639067 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.639037 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkt2b\" (UniqueName: \"kubernetes.io/projected/4b5e6cdc-d6e2-460f-9329-6d5618ef1aed-kube-api-access-lkt2b\") pod \"openshift-lws-operator-fd8ccff4c-qqvf9\" (UID: \"4b5e6cdc-d6e2-460f-9329-6d5618ef1aed\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" Jul 20 15:54:55.639212 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.639073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b5e6cdc-d6e2-460f-9329-6d5618ef1aed-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qqvf9\" (UID: \"4b5e6cdc-d6e2-460f-9329-6d5618ef1aed\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" Jul 20 15:54:55.740654 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.740560 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkt2b\" (UniqueName: \"kubernetes.io/projected/4b5e6cdc-d6e2-460f-9329-6d5618ef1aed-kube-api-access-lkt2b\") pod \"openshift-lws-operator-fd8ccff4c-qqvf9\" (UID: \"4b5e6cdc-d6e2-460f-9329-6d5618ef1aed\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" Jul 20 15:54:55.740654 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.740604 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b5e6cdc-d6e2-460f-9329-6d5618ef1aed-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qqvf9\" (UID: \"4b5e6cdc-d6e2-460f-9329-6d5618ef1aed\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" Jul 20 15:54:55.741029 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.741010 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b5e6cdc-d6e2-460f-9329-6d5618ef1aed-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qqvf9\" (UID: \"4b5e6cdc-d6e2-460f-9329-6d5618ef1aed\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" Jul 20 15:54:55.749177 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.749144 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkt2b\" (UniqueName: \"kubernetes.io/projected/4b5e6cdc-d6e2-460f-9329-6d5618ef1aed-kube-api-access-lkt2b\") pod \"openshift-lws-operator-fd8ccff4c-qqvf9\" (UID: \"4b5e6cdc-d6e2-460f-9329-6d5618ef1aed\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" Jul 20 15:54:55.757826 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.757800 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-gpxqn" event={"ID":"b6bd290a-ada8-40a6-b135-727cea6d4768","Type":"ContainerStarted","Data":"f6795b5842c62bdcab960edd2c5413507a2a5649a5b5b1c05ec4532c520be498"} Jul 20 15:54:55.773947 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.773897 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-gpxqn" podStartSLOduration=0.855990323 podStartE2EDuration="3.773880982s" totalImagesPullingTime="2.917890659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:54:55.772917351 +0000 UTC m=+233.951594424" watchObservedRunningTime="2026-07-20 15:54:55.773880982 +0000 UTC m=+233.952558054" Jul 20 15:54:55.783480 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.783446 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" Jul 20 15:54:55.928806 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:55.928776 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9"] Jul 20 15:54:55.932100 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:54:55.932073 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b5e6cdc_d6e2_460f_9329_6d5618ef1aed.slice/crio-56395172a1c8ff50c3124e3dc4fe67ff31a596b82edde8d4a6906c7e6778de1b WatchSource:0}: Error finding container 56395172a1c8ff50c3124e3dc4fe67ff31a596b82edde8d4a6906c7e6778de1b: Status 404 returned error can't find the container with id 56395172a1c8ff50c3124e3dc4fe67ff31a596b82edde8d4a6906c7e6778de1b Jul 20 15:54:56.765920 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:56.765867 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" event={"ID":"4b5e6cdc-d6e2-460f-9329-6d5618ef1aed","Type":"ContainerStarted","Data":"56395172a1c8ff50c3124e3dc4fe67ff31a596b82edde8d4a6906c7e6778de1b"} Jul 20 15:54:58.776792 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:58.776697 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" event={"ID":"4b5e6cdc-d6e2-460f-9329-6d5618ef1aed","Type":"ContainerStarted","Data":"97091b2f5608188fe1bcc00409c0363f00bdf63fef2a44b3cd1bb150885452c3"} Jul 20 15:54:58.804549 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:54:58.804496 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qqvf9" podStartSLOduration=1.312646375 podStartE2EDuration="3.804477392s" totalImagesPullingTime="2.491831017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:54:58.802028593 +0000 UTC m=+236.980705668" watchObservedRunningTime="2026-07-20 15:54:58.804477392 +0000 UTC m=+236.983154464" Jul 20 15:55:19.524771 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.524740 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x"] Jul 20 15:55:19.532950 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.532929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.539861 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.539836 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 15:55:19.539964 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.539874 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 15:55:19.542279 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.542260 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ft7hk\"" Jul 20 15:55:19.542354 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.542289 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 15:55:19.542400 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.542376 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 15:55:19.549287 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.549262 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x"] Jul 20 15:55:19.637195 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.637165 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/361566c9-8d7d-4e58-9ac5-d128287f4adc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.637318 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.637222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhnh7\" (UniqueName: \"kubernetes.io/projected/361566c9-8d7d-4e58-9ac5-d128287f4adc-kube-api-access-qhnh7\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.637318 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.637305 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/361566c9-8d7d-4e58-9ac5-d128287f4adc-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.637392 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.637342 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/361566c9-8d7d-4e58-9ac5-d128287f4adc-tmp\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.637392 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.637370 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/361566c9-8d7d-4e58-9ac5-d128287f4adc-manifests\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.738426 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.738402 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/361566c9-8d7d-4e58-9ac5-d128287f4adc-manifests\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.738543 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.738438 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/361566c9-8d7d-4e58-9ac5-d128287f4adc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.738543 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.738503 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhnh7\" (UniqueName: \"kubernetes.io/projected/361566c9-8d7d-4e58-9ac5-d128287f4adc-kube-api-access-qhnh7\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.738631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.738571 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/361566c9-8d7d-4e58-9ac5-d128287f4adc-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.738631 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.738607 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/361566c9-8d7d-4e58-9ac5-d128287f4adc-tmp\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.738876 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.738854 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/361566c9-8d7d-4e58-9ac5-d128287f4adc-manifests\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.740816 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.740790 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/361566c9-8d7d-4e58-9ac5-d128287f4adc-tmp\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.741027 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.741010 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/361566c9-8d7d-4e58-9ac5-d128287f4adc-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.741092 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.741070 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/361566c9-8d7d-4e58-9ac5-d128287f4adc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.750883 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.750858 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhnh7\" (UniqueName: \"kubernetes.io/projected/361566c9-8d7d-4e58-9ac5-d128287f4adc-kube-api-access-qhnh7\") pod \"opendatahub-operator-controller-manager-5b76897bc6-st97x\" (UID: \"361566c9-8d7d-4e58-9ac5-d128287f4adc\") " pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.843915 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.843884 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:19.977886 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:19.977862 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x"] Jul 20 15:55:19.980168 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:55:19.980140 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361566c9_8d7d_4e58_9ac5_d128287f4adc.slice/crio-399ddd33c4e4e6c3e0f851025d5c8e8b252e1cf97079a61ad85eb1db5299ae73 WatchSource:0}: Error finding container 399ddd33c4e4e6c3e0f851025d5c8e8b252e1cf97079a61ad85eb1db5299ae73: Status 404 returned error can't find the container with id 399ddd33c4e4e6c3e0f851025d5c8e8b252e1cf97079a61ad85eb1db5299ae73 Jul 20 15:55:20.869942 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:20.869900 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" event={"ID":"361566c9-8d7d-4e58-9ac5-d128287f4adc","Type":"ContainerStarted","Data":"399ddd33c4e4e6c3e0f851025d5c8e8b252e1cf97079a61ad85eb1db5299ae73"} Jul 20 15:55:23.885023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:23.884988 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" event={"ID":"361566c9-8d7d-4e58-9ac5-d128287f4adc","Type":"ContainerStarted","Data":"eaf54ea248043da7981b3459bfb357865afec71c7428ccdaf9a47986126ca2ea"} Jul 20 15:55:24.891326 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:24.891288 2559 generic.go:356] "Generic (PLEG): container finished" podID="361566c9-8d7d-4e58-9ac5-d128287f4adc" containerID="eaf54ea248043da7981b3459bfb357865afec71c7428ccdaf9a47986126ca2ea" exitCode=0 Jul 20 15:55:24.891766 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:24.891336 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" event={"ID":"361566c9-8d7d-4e58-9ac5-d128287f4adc","Type":"ContainerDied","Data":"eaf54ea248043da7981b3459bfb357865afec71c7428ccdaf9a47986126ca2ea"} Jul 20 15:55:25.897404 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:25.897371 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" event={"ID":"361566c9-8d7d-4e58-9ac5-d128287f4adc","Type":"ContainerStarted","Data":"ebfb618532d8e2b6ffcdad0a1fce63b5bc0d0aea52b3f801d0423179e2de5082"} Jul 20 15:55:25.897837 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:25.897534 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:25.945926 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:25.945882 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" podStartSLOduration=2.698241144 podStartE2EDuration="6.945867147s" totalImagesPullingTime="3.232314976s" totalInitContainerRuntime="1.015311027s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:55:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:55:25.944640237 +0000 UTC m=+264.123317342" watchObservedRunningTime="2026-07-20 15:55:25.945867147 +0000 UTC m=+264.124544219" Jul 20 15:55:36.904706 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:36.904654 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b76897bc6-st97x" Jul 20 15:55:47.597352 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.597317 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t"] Jul 20 15:55:47.605548 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.605526 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.609563 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.609544 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 15:55:47.609563 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.609553 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-pfx5x\"" Jul 20 15:55:47.612416 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.612219 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 15:55:47.612416 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.612286 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 15:55:47.619273 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.619245 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t"] Jul 20 15:55:47.788835 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.788795 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-cert\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.789023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.788855 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-manager-config\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.789023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.788933 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-metrics-cert\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.789023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.788977 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvg8l\" (UniqueName: \"kubernetes.io/projected/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-kube-api-access-dvg8l\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.889708 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.889599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-metrics-cert\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.889708 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.889640 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvg8l\" (UniqueName: \"kubernetes.io/projected/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-kube-api-access-dvg8l\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.889909 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.889752 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-cert\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.889909 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.889809 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-manager-config\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.890449 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.890426 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-manager-config\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.892227 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.892203 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-cert\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.892227 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.892222 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-metrics-cert\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:47.923878 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:47.923846 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvg8l\" (UniqueName: \"kubernetes.io/projected/bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e-kube-api-access-dvg8l\") pod \"lws-controller-manager-8c9c5dcb7-rdc5t\" (UID: \"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e\") " pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:48.215348 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:48.215262 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:48.357547 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:48.357514 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t"] Jul 20 15:55:48.360457 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:55:48.360422 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7bf3a7_18b1_4ebf_8764_68cf6e88b43e.slice/crio-08753913df63e95c0712dd7b0d69f860b4eefec70e39b34ed094221ae7b16547 WatchSource:0}: Error finding container 08753913df63e95c0712dd7b0d69f860b4eefec70e39b34ed094221ae7b16547: Status 404 returned error can't find the container with id 08753913df63e95c0712dd7b0d69f860b4eefec70e39b34ed094221ae7b16547 Jul 20 15:55:49.000335 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:49.000301 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" event={"ID":"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e","Type":"ContainerStarted","Data":"08753913df63e95c0712dd7b0d69f860b4eefec70e39b34ed094221ae7b16547"} Jul 20 15:55:50.008079 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.008048 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" event={"ID":"bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e","Type":"ContainerStarted","Data":"dd14bdd0277f87434ed9ef20a0f3f3aaf2013ecd850072494762f7a29d803570"} Jul 20 15:55:50.008486 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.008152 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:55:50.056230 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.056181 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" podStartSLOduration=1.566388308 podStartE2EDuration="3.056164848s" totalImagesPullingTime="1.48977654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:55:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:55:50.051197956 +0000 UTC m=+288.229875030" watchObservedRunningTime="2026-07-20 15:55:50.056164848 +0000 UTC m=+288.234841977" Jul 20 15:55:50.228917 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.228838 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p"] Jul 20 15:55:50.232378 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.232362 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.237136 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.237113 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 15:55:50.237136 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.237113 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 20 15:55:50.239752 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.239723 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 15:55:50.240412 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.240398 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m84xq\"" Jul 20 15:55:50.241060 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.241044 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 20 15:55:50.251788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.251769 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p"] Jul 20 15:55:50.309203 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.309171 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbz4t\" (UniqueName: \"kubernetes.io/projected/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-kube-api-access-nbz4t\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.309374 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.309221 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tmp\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.309374 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.309298 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tls-certs\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.410036 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.410000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tls-certs\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.410203 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.410051 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbz4t\" (UniqueName: \"kubernetes.io/projected/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-kube-api-access-nbz4t\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.410203 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.410095 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tmp\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.410203 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:55:50.410150 2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 20 15:55:50.410305 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:55:50.410210 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tls-certs podName:97d46b2f-d3d7-45b5-8ff2-5f9005cdb135 nodeName:}" failed. No retries permitted until 2026-07-20 15:55:50.910193727 +0000 UTC m=+289.088870778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tls-certs") pod "kube-auth-proxy-bc6c54cc6-8zs5p" (UID: "97d46b2f-d3d7-45b5-8ff2-5f9005cdb135") : secret "kube-auth-proxy-tls" not found Jul 20 15:55:50.412426 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.412399 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tmp\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.420053 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.420030 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbz4t\" (UniqueName: \"kubernetes.io/projected/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-kube-api-access-nbz4t\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.914122 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.914083 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tls-certs\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:50.916816 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:50.916786 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97d46b2f-d3d7-45b5-8ff2-5f9005cdb135-tls-certs\") pod \"kube-auth-proxy-bc6c54cc6-8zs5p\" (UID: \"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135\") " pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:51.141728 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:51.141695 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" Jul 20 15:55:51.310881 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:51.310850 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p"] Jul 20 15:55:51.314126 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:55:51.314106 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d46b2f_d3d7_45b5_8ff2_5f9005cdb135.slice/crio-cf5c4c7caf09fe90f433abf4e48d40be3a04a52a46e98852115f4adc166c2e4b WatchSource:0}: Error finding container cf5c4c7caf09fe90f433abf4e48d40be3a04a52a46e98852115f4adc166c2e4b: Status 404 returned error can't find the container with id cf5c4c7caf09fe90f433abf4e48d40be3a04a52a46e98852115f4adc166c2e4b Jul 20 15:55:52.020994 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:52.020955 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" event={"ID":"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135","Type":"ContainerStarted","Data":"cf5c4c7caf09fe90f433abf4e48d40be3a04a52a46e98852115f4adc166c2e4b"} Jul 20 15:55:54.032806 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:54.032769 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" event={"ID":"97d46b2f-d3d7-45b5-8ff2-5f9005cdb135","Type":"ContainerStarted","Data":"abca8f7613c504ab7d05d09349ae94a18b33c364ba129605900443244546f583"} Jul 20 15:55:54.060806 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:55:54.060760 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-bc6c54cc6-8zs5p" podStartSLOduration=1.4985231319999999 podStartE2EDuration="4.060745053s" totalImagesPullingTime="2.562221921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:55:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:55:54.05993747 +0000 UTC m=+292.238614546" watchObservedRunningTime="2026-07-20 15:55:54.060745053 +0000 UTC m=+292.239422126" Jul 20 15:56:01.014808 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:56:01.014778 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8c9c5dcb7-rdc5t" Jul 20 15:56:02.181434 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:56:02.181406 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 15:57:19.388696 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.388642 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s"] Jul 20 15:57:19.392191 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.392173 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" Jul 20 15:57:19.394716 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.394695 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 15:57:19.394860 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.394751 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kzcqs\"" Jul 20 15:57:19.395253 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.395238 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 15:57:19.404968 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.404936 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s"] Jul 20 15:57:19.419238 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.419201 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngrlr\" (UniqueName: \"kubernetes.io/projected/cc4f8f26-f770-43bb-b5e9-783cddb9f88f-kube-api-access-ngrlr\") pod \"authorino-operator-75cc8d5cb7-qsj2s\" (UID: \"cc4f8f26-f770-43bb-b5e9-783cddb9f88f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" Jul 20 15:57:19.520475 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.520439 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrlr\" (UniqueName: \"kubernetes.io/projected/cc4f8f26-f770-43bb-b5e9-783cddb9f88f-kube-api-access-ngrlr\") pod \"authorino-operator-75cc8d5cb7-qsj2s\" (UID: \"cc4f8f26-f770-43bb-b5e9-783cddb9f88f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" Jul 20 15:57:19.533399 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.533365 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngrlr\" (UniqueName: \"kubernetes.io/projected/cc4f8f26-f770-43bb-b5e9-783cddb9f88f-kube-api-access-ngrlr\") pod \"authorino-operator-75cc8d5cb7-qsj2s\" (UID: \"cc4f8f26-f770-43bb-b5e9-783cddb9f88f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" Jul 20 15:57:19.701964 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.701877 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" Jul 20 15:57:19.847349 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.847318 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s"] Jul 20 15:57:19.850417 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:57:19.850391 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc4f8f26_f770_43bb_b5e9_783cddb9f88f.slice/crio-2d5fee955c17d010e2963c17bd8dc370d47a81b366616b8a6f7a6530fbdfa449 WatchSource:0}: Error finding container 2d5fee955c17d010e2963c17bd8dc370d47a81b366616b8a6f7a6530fbdfa449: Status 404 returned error can't find the container with id 2d5fee955c17d010e2963c17bd8dc370d47a81b366616b8a6f7a6530fbdfa449 Jul 20 15:57:19.852508 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.852492 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 15:57:19.964483 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.964398 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7995cbcd6-s6j9x"] Jul 20 15:57:19.969023 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.968999 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:19.985869 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:19.985843 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7995cbcd6-s6j9x"] Jul 20 15:57:20.025146 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.025121 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/346cca8c-b2d3-427a-afdb-255526d346fa-console-serving-cert\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.025146 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.025148 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-console-config\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.025322 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.025164 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-service-ca\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.025322 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.025185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-oauth-serving-cert\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.025322 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.025206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-trusted-ca-bundle\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.025322 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.025278 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ltj7\" (UniqueName: \"kubernetes.io/projected/346cca8c-b2d3-427a-afdb-255526d346fa-kube-api-access-9ltj7\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.025322 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.025318 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/346cca8c-b2d3-427a-afdb-255526d346fa-console-oauth-config\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.126569 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.126540 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ltj7\" (UniqueName: \"kubernetes.io/projected/346cca8c-b2d3-427a-afdb-255526d346fa-kube-api-access-9ltj7\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.126734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.126589 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/346cca8c-b2d3-427a-afdb-255526d346fa-console-oauth-config\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.126734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.126632 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/346cca8c-b2d3-427a-afdb-255526d346fa-console-serving-cert\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.126734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.126646 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-console-config\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.126734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.126662 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-service-ca\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.126734 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.126712 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-oauth-serving-cert\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.126963 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.126740 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-trusted-ca-bundle\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.127535 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.127508 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-oauth-serving-cert\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.127646 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.127516 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-console-config\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.127646 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.127593 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-trusted-ca-bundle\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.127805 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.127780 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/346cca8c-b2d3-427a-afdb-255526d346fa-service-ca\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.129125 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.129101 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/346cca8c-b2d3-427a-afdb-255526d346fa-console-oauth-config\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.129213 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.129192 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/346cca8c-b2d3-427a-afdb-255526d346fa-console-serving-cert\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.138795 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.138776 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ltj7\" (UniqueName: \"kubernetes.io/projected/346cca8c-b2d3-427a-afdb-255526d346fa-kube-api-access-9ltj7\") pod \"console-7995cbcd6-s6j9x\" (UID: \"346cca8c-b2d3-427a-afdb-255526d346fa\") " pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.278094 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.278022 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:20.417408 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.417373 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7995cbcd6-s6j9x"] Jul 20 15:57:20.420431 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:57:20.420401 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346cca8c_b2d3_427a_afdb_255526d346fa.slice/crio-031b7c22a20865e94262b5931c9aa1be8634d7169f1cb03160ae00149835912e WatchSource:0}: Error finding container 031b7c22a20865e94262b5931c9aa1be8634d7169f1cb03160ae00149835912e: Status 404 returned error can't find the container with id 031b7c22a20865e94262b5931c9aa1be8634d7169f1cb03160ae00149835912e Jul 20 15:57:20.430839 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.430812 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7995cbcd6-s6j9x" event={"ID":"346cca8c-b2d3-427a-afdb-255526d346fa","Type":"ContainerStarted","Data":"031b7c22a20865e94262b5931c9aa1be8634d7169f1cb03160ae00149835912e"} Jul 20 15:57:20.432045 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:20.432009 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" event={"ID":"cc4f8f26-f770-43bb-b5e9-783cddb9f88f","Type":"ContainerStarted","Data":"2d5fee955c17d010e2963c17bd8dc370d47a81b366616b8a6f7a6530fbdfa449"} Jul 20 15:57:21.439700 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:21.439633 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7995cbcd6-s6j9x" event={"ID":"346cca8c-b2d3-427a-afdb-255526d346fa","Type":"ContainerStarted","Data":"c29b6296be827449f9c6f52ec210eb3242ea843cd03a56d689d7c1188e61eb82"} Jul 20 15:57:21.465627 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:21.465269 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7995cbcd6-s6j9x" podStartSLOduration=2.465249724 podStartE2EDuration="2.465249724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:57:21.464946594 +0000 UTC m=+379.643623682" watchObservedRunningTime="2026-07-20 15:57:21.465249724 +0000 UTC m=+379.643926798" Jul 20 15:57:22.452209 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:22.452105 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" event={"ID":"cc4f8f26-f770-43bb-b5e9-783cddb9f88f","Type":"ContainerStarted","Data":"7df9e394d56bbc9f7b3a7f4294163e6a609a14e9f094488cab99205f54461daf"} Jul 20 15:57:22.452787 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:22.452326 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" Jul 20 15:57:22.476141 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:22.476094 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" podStartSLOduration=1.16396098 podStartE2EDuration="3.476081315s" totalImagesPullingTime="2.312120335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:57:22.473877877 +0000 UTC m=+380.652554950" watchObservedRunningTime="2026-07-20 15:57:22.476081315 +0000 UTC m=+380.654758389" Jul 20 15:57:30.278953 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:30.278909 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:30.279410 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:30.279003 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:30.283849 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:30.283824 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:30.494406 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:30.494378 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7995cbcd6-s6j9x" Jul 20 15:57:30.679343 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:30.679312 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6df75c8764-2rzhf"] Jul 20 15:57:33.459650 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:33.459619 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qsj2s" Jul 20 15:57:55.699408 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:55.699333 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6df75c8764-2rzhf" podUID="c26b38cd-d4ff-43da-8c17-33487944db68" containerName="console" containerID="cri-o://b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56" gracePeriod=15 Jul 20 15:57:55.948576 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:55.948547 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:57:56.057288 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057257 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-trusted-ca-bundle\") pod \"c26b38cd-d4ff-43da-8c17-33487944db68\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " Jul 20 15:57:56.057456 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057299 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-oauth-serving-cert\") pod \"c26b38cd-d4ff-43da-8c17-33487944db68\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " Jul 20 15:57:56.057456 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057322 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxk2g\" (UniqueName: \"kubernetes.io/projected/c26b38cd-d4ff-43da-8c17-33487944db68-kube-api-access-dxk2g\") pod \"c26b38cd-d4ff-43da-8c17-33487944db68\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " Jul 20 15:57:56.057456 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057360 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-serving-cert\") pod \"c26b38cd-d4ff-43da-8c17-33487944db68\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " Jul 20 15:57:56.057456 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057380 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-oauth-config\") pod \"c26b38cd-d4ff-43da-8c17-33487944db68\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " Jul 20 15:57:56.057456 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057407 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-console-config\") pod \"c26b38cd-d4ff-43da-8c17-33487944db68\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " Jul 20 15:57:56.057721 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057499 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-service-ca\") pod \"c26b38cd-d4ff-43da-8c17-33487944db68\" (UID: \"c26b38cd-d4ff-43da-8c17-33487944db68\") " Jul 20 15:57:56.057788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057709 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c26b38cd-d4ff-43da-8c17-33487944db68" (UID: "c26b38cd-d4ff-43da-8c17-33487944db68"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:57:56.057788 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057722 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c26b38cd-d4ff-43da-8c17-33487944db68" (UID: "c26b38cd-d4ff-43da-8c17-33487944db68"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:57:56.057895 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057819 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-trusted-ca-bundle\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:57:56.057895 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.057839 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-oauth-serving-cert\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:57:56.058049 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.058009 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-console-config" (OuterVolumeSpecName: "console-config") pod "c26b38cd-d4ff-43da-8c17-33487944db68" (UID: "c26b38cd-d4ff-43da-8c17-33487944db68"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:57:56.058328 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.058132 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-service-ca" (OuterVolumeSpecName: "service-ca") pod "c26b38cd-d4ff-43da-8c17-33487944db68" (UID: "c26b38cd-d4ff-43da-8c17-33487944db68"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:57:56.059853 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.059826 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c26b38cd-d4ff-43da-8c17-33487944db68" (UID: "c26b38cd-d4ff-43da-8c17-33487944db68"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:57:56.059853 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.059838 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c26b38cd-d4ff-43da-8c17-33487944db68-kube-api-access-dxk2g" (OuterVolumeSpecName: "kube-api-access-dxk2g") pod "c26b38cd-d4ff-43da-8c17-33487944db68" (UID: "c26b38cd-d4ff-43da-8c17-33487944db68"). InnerVolumeSpecName "kube-api-access-dxk2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:57:56.059853 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.059843 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c26b38cd-d4ff-43da-8c17-33487944db68" (UID: "c26b38cd-d4ff-43da-8c17-33487944db68"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:57:56.159256 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.159227 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-service-ca\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:57:56.159256 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.159253 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxk2g\" (UniqueName: \"kubernetes.io/projected/c26b38cd-d4ff-43da-8c17-33487944db68-kube-api-access-dxk2g\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:57:56.159256 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.159263 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-serving-cert\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:57:56.159470 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.159273 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26b38cd-d4ff-43da-8c17-33487944db68-console-oauth-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:57:56.159470 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.159281 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26b38cd-d4ff-43da-8c17-33487944db68-console-config\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:57:56.613596 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.613512 2559 generic.go:356] "Generic (PLEG): container finished" podID="c26b38cd-d4ff-43da-8c17-33487944db68" containerID="b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56" exitCode=0 Jul 20 15:57:56.613596 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.613558 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6df75c8764-2rzhf" Jul 20 15:57:56.613815 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.613600 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6df75c8764-2rzhf" event={"ID":"c26b38cd-d4ff-43da-8c17-33487944db68","Type":"ContainerDied","Data":"b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56"} Jul 20 15:57:56.613815 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.613653 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6df75c8764-2rzhf" event={"ID":"c26b38cd-d4ff-43da-8c17-33487944db68","Type":"ContainerDied","Data":"363cf8e00e5716c6705ad87350b474d408ff33573d321218dd84551da5c7ac6d"} Jul 20 15:57:56.613815 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.613698 2559 scope.go:117] "RemoveContainer" containerID="b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56" Jul 20 15:57:56.623193 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.623172 2559 scope.go:117] "RemoveContainer" containerID="b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56" Jul 20 15:57:56.625210 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:57:56.625184 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56\": container with ID starting with b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56 not found: ID does not exist" containerID="b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56" Jul 20 15:57:56.625336 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.625216 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56"} err="failed to get container status \"b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56\": rpc error: code = NotFound desc = could not find container \"b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56\": container with ID starting with b75ec9c5ffe5ae397482bbd4625d6c248724dbcf0050129473f4a486cc7bef56 not found: ID does not exist" Jul 20 15:57:56.634982 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.634960 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6df75c8764-2rzhf"] Jul 20 15:57:56.638928 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:56.638907 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6df75c8764-2rzhf"] Jul 20 15:57:58.317434 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:57:58.317393 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c26b38cd-d4ff-43da-8c17-33487944db68" path="/var/lib/kubelet/pods/c26b38cd-d4ff-43da-8c17-33487944db68/volumes" Jul 20 15:58:17.717850 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.717814 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-g8zwk"] Jul 20 15:58:17.719282 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.719255 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c26b38cd-d4ff-43da-8c17-33487944db68" containerName="console" Jul 20 15:58:17.719412 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.719402 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c26b38cd-d4ff-43da-8c17-33487944db68" containerName="console" Jul 20 15:58:17.719717 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.719703 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c26b38cd-d4ff-43da-8c17-33487944db68" containerName="console" Jul 20 15:58:17.725877 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.725855 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g8zwk" Jul 20 15:58:17.728698 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.728651 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jlc4r\"" Jul 20 15:58:17.730394 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.730371 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-g8zwk"] Jul 20 15:58:17.842471 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.842438 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7kvn\" (UniqueName: \"kubernetes.io/projected/ed752298-bd54-4696-863b-3dd4f8769b7e-kube-api-access-g7kvn\") pod \"authorino-58cd955675-g8zwk\" (UID: \"ed752298-bd54-4696-863b-3dd4f8769b7e\") " pod="kuadrant-system/authorino-58cd955675-g8zwk" Jul 20 15:58:17.943602 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.943569 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7kvn\" (UniqueName: \"kubernetes.io/projected/ed752298-bd54-4696-863b-3dd4f8769b7e-kube-api-access-g7kvn\") pod \"authorino-58cd955675-g8zwk\" (UID: \"ed752298-bd54-4696-863b-3dd4f8769b7e\") " pod="kuadrant-system/authorino-58cd955675-g8zwk" Jul 20 15:58:17.953119 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:17.953085 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7kvn\" (UniqueName: \"kubernetes.io/projected/ed752298-bd54-4696-863b-3dd4f8769b7e-kube-api-access-g7kvn\") pod \"authorino-58cd955675-g8zwk\" (UID: \"ed752298-bd54-4696-863b-3dd4f8769b7e\") " pod="kuadrant-system/authorino-58cd955675-g8zwk" Jul 20 15:58:18.035995 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:18.035967 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g8zwk" Jul 20 15:58:18.176579 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:18.176546 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-g8zwk"] Jul 20 15:58:18.180069 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:58:18.180037 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded752298_bd54_4696_863b_3dd4f8769b7e.slice/crio-d44ad3d92a4caa86d0dddb499232e10c966e533d63dd0cefede027383a5f6ea1 WatchSource:0}: Error finding container d44ad3d92a4caa86d0dddb499232e10c966e533d63dd0cefede027383a5f6ea1: Status 404 returned error can't find the container with id d44ad3d92a4caa86d0dddb499232e10c966e533d63dd0cefede027383a5f6ea1 Jul 20 15:58:18.714235 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:18.714202 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g8zwk" event={"ID":"ed752298-bd54-4696-863b-3dd4f8769b7e","Type":"ContainerStarted","Data":"d44ad3d92a4caa86d0dddb499232e10c966e533d63dd0cefede027383a5f6ea1"} Jul 20 15:58:21.732810 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:21.732719 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g8zwk" event={"ID":"ed752298-bd54-4696-863b-3dd4f8769b7e","Type":"ContainerStarted","Data":"be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74"} Jul 20 15:58:21.757034 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:21.756964 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-g8zwk" podStartSLOduration=1.5099499939999999 podStartE2EDuration="4.75694246s" totalImagesPullingTime="3.246992466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:58:21.75671898 +0000 UTC m=+439.935396055" watchObservedRunningTime="2026-07-20 15:58:21.75694246 +0000 UTC m=+439.935619534" Jul 20 15:58:22.968331 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:22.968252 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-g8zwk"] Jul 20 15:58:23.742648 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:23.742592 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-g8zwk" podUID="ed752298-bd54-4696-863b-3dd4f8769b7e" containerName="authorino" containerID="cri-o://be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74" gracePeriod=30 Jul 20 15:58:23.986234 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:23.986211 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g8zwk" Jul 20 15:58:24.096991 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.096961 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7kvn\" (UniqueName: \"kubernetes.io/projected/ed752298-bd54-4696-863b-3dd4f8769b7e-kube-api-access-g7kvn\") pod \"ed752298-bd54-4696-863b-3dd4f8769b7e\" (UID: \"ed752298-bd54-4696-863b-3dd4f8769b7e\") " Jul 20 15:58:24.099277 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.099248 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed752298-bd54-4696-863b-3dd4f8769b7e-kube-api-access-g7kvn" (OuterVolumeSpecName: "kube-api-access-g7kvn") pod "ed752298-bd54-4696-863b-3dd4f8769b7e" (UID: "ed752298-bd54-4696-863b-3dd4f8769b7e"). InnerVolumeSpecName "kube-api-access-g7kvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:58:24.197936 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.197903 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7kvn\" (UniqueName: \"kubernetes.io/projected/ed752298-bd54-4696-863b-3dd4f8769b7e-kube-api-access-g7kvn\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 15:58:24.748893 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.748859 2559 generic.go:356] "Generic (PLEG): container finished" podID="ed752298-bd54-4696-863b-3dd4f8769b7e" containerID="be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74" exitCode=0 Jul 20 15:58:24.748893 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.748897 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g8zwk" Jul 20 15:58:24.749137 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.748934 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g8zwk" event={"ID":"ed752298-bd54-4696-863b-3dd4f8769b7e","Type":"ContainerDied","Data":"be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74"} Jul 20 15:58:24.749137 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.748966 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g8zwk" event={"ID":"ed752298-bd54-4696-863b-3dd4f8769b7e","Type":"ContainerDied","Data":"d44ad3d92a4caa86d0dddb499232e10c966e533d63dd0cefede027383a5f6ea1"} Jul 20 15:58:24.749137 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.748982 2559 scope.go:117] "RemoveContainer" containerID="be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74" Jul 20 15:58:24.757205 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.757186 2559 scope.go:117] "RemoveContainer" containerID="be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74" Jul 20 15:58:24.757502 ip-10-0-142-55 kubenswrapper[2559]: E0720 15:58:24.757484 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74\": container with ID starting with be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74 not found: ID does not exist" containerID="be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74" Jul 20 15:58:24.757560 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.757511 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74"} err="failed to get container status \"be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74\": rpc error: code = NotFound desc = could not find container \"be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74\": container with ID starting with be4eafece1cb43b53fcb7a3ecdf8c77f68f6f7f363cc376337fbe0ac0ab7ae74 not found: ID does not exist" Jul 20 15:58:24.765372 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.765350 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-g8zwk"] Jul 20 15:58:24.767370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:24.767350 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-g8zwk"] Jul 20 15:58:26.317825 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:58:26.317787 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed752298-bd54-4696-863b-3dd4f8769b7e" path="/var/lib/kubelet/pods/ed752298-bd54-4696-863b-3dd4f8769b7e/volumes" Jul 20 15:59:34.822320 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.822289 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-d9j42"] Jul 20 15:59:34.822800 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.822653 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed752298-bd54-4696-863b-3dd4f8769b7e" containerName="authorino" Jul 20 15:59:34.822800 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.822665 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed752298-bd54-4696-863b-3dd4f8769b7e" containerName="authorino" Jul 20 15:59:34.822800 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.822754 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed752298-bd54-4696-863b-3dd4f8769b7e" containerName="authorino" Jul 20 15:59:34.825573 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.825555 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:34.827717 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.827694 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 20 15:59:34.827832 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.827764 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-626wc\"" Jul 20 15:59:34.836249 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.836220 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-d9j42"] Jul 20 15:59:34.933568 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.933523 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbbe3a31-924b-4e92-b0fc-948646b5336a-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-d9j42\" (UID: \"fbbe3a31-924b-4e92-b0fc-948646b5336a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:34.933754 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:34.933603 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtzzs\" (UniqueName: \"kubernetes.io/projected/fbbe3a31-924b-4e92-b0fc-948646b5336a-kube-api-access-wtzzs\") pod \"payload-pre-processing-7677b774bb-d9j42\" (UID: \"fbbe3a31-924b-4e92-b0fc-948646b5336a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:35.034956 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:35.034910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbbe3a31-924b-4e92-b0fc-948646b5336a-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-d9j42\" (UID: \"fbbe3a31-924b-4e92-b0fc-948646b5336a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:35.035123 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:35.034971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtzzs\" (UniqueName: \"kubernetes.io/projected/fbbe3a31-924b-4e92-b0fc-948646b5336a-kube-api-access-wtzzs\") pod \"payload-pre-processing-7677b774bb-d9j42\" (UID: \"fbbe3a31-924b-4e92-b0fc-948646b5336a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:35.035587 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:35.035569 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbbe3a31-924b-4e92-b0fc-948646b5336a-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-d9j42\" (UID: \"fbbe3a31-924b-4e92-b0fc-948646b5336a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:35.043201 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:35.043177 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtzzs\" (UniqueName: \"kubernetes.io/projected/fbbe3a31-924b-4e92-b0fc-948646b5336a-kube-api-access-wtzzs\") pod \"payload-pre-processing-7677b774bb-d9j42\" (UID: \"fbbe3a31-924b-4e92-b0fc-948646b5336a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:35.135904 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:35.135827 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:35.274648 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:35.274622 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-d9j42"] Jul 20 15:59:35.277022 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:59:35.276991 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbe3a31_924b_4e92_b0fc_948646b5336a.slice/crio-86b2ea626af2bf8d555e579b3f6ef898925eab5773d75488965be105ec90d152 WatchSource:0}: Error finding container 86b2ea626af2bf8d555e579b3f6ef898925eab5773d75488965be105ec90d152: Status 404 returned error can't find the container with id 86b2ea626af2bf8d555e579b3f6ef898925eab5773d75488965be105ec90d152 Jul 20 15:59:36.084828 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:36.084790 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" event={"ID":"fbbe3a31-924b-4e92-b0fc-948646b5336a","Type":"ContainerStarted","Data":"86b2ea626af2bf8d555e579b3f6ef898925eab5773d75488965be105ec90d152"} Jul 20 15:59:37.091248 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:37.091219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" event={"ID":"fbbe3a31-924b-4e92-b0fc-948646b5336a","Type":"ContainerStarted","Data":"0255a33d6f0eb6011af73d0f1b9bb5684458e5c70732b159ea1d3463920367db"} Jul 20 15:59:37.091621 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:37.091309 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:37.108777 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:37.108693 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" podStartSLOduration=1.5426456229999999 podStartE2EDuration="3.108660082s" totalImagesPullingTime="1.566014459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:59:37.10720498 +0000 UTC m=+515.285882077" watchObservedRunningTime="2026-07-20 15:59:37.108660082 +0000 UTC m=+515.287337158" Jul 20 15:59:48.099290 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:48.099258 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-d9j42" Jul 20 15:59:52.798291 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.798220 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd"] Jul 20 15:59:52.801730 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.801714 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.804785 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.804760 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lpsx4\"" Jul 20 15:59:52.804910 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.804869 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 15:59:52.804910 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.804896 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 20 15:59:52.804910 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.804901 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 15:59:52.813984 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.813962 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd"] Jul 20 15:59:52.894179 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.894143 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74751610-3411-43d1-ace7-1a964f330734-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.894370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.894183 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.894370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.894212 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlw67\" (UniqueName: \"kubernetes.io/projected/74751610-3411-43d1-ace7-1a964f330734-kube-api-access-tlw67\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.894370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.894261 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.894370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.894319 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.894370 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.894353 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.995566 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.995529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74751610-3411-43d1-ace7-1a964f330734-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.995566 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.995566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.995853 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.995586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlw67\" (UniqueName: \"kubernetes.io/projected/74751610-3411-43d1-ace7-1a964f330734-kube-api-access-tlw67\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.995853 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.995621 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.995853 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.995651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.995853 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.995668 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.996061 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.996023 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.996102 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.996066 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.996135 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.996105 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.998035 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.998015 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74751610-3411-43d1-ace7-1a964f330734-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:52.998310 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:52.998289 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74751610-3411-43d1-ace7-1a964f330734-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:53.006129 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:53.006108 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlw67\" (UniqueName: \"kubernetes.io/projected/74751610-3411-43d1-ace7-1a964f330734-kube-api-access-tlw67\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd\" (UID: \"74751610-3411-43d1-ace7-1a964f330734\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:53.111882 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:53.111796 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 15:59:53.248319 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:53.248255 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd"] Jul 20 15:59:53.250739 ip-10-0-142-55 kubenswrapper[2559]: W0720 15:59:53.250708 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74751610_3411_43d1_ace7_1a964f330734.slice/crio-1f24f1db5e09c3d77f04658ed0aa0684e9dc405505881cb8d20339280e354f3d WatchSource:0}: Error finding container 1f24f1db5e09c3d77f04658ed0aa0684e9dc405505881cb8d20339280e354f3d: Status 404 returned error can't find the container with id 1f24f1db5e09c3d77f04658ed0aa0684e9dc405505881cb8d20339280e354f3d Jul 20 15:59:54.178745 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:54.178711 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" event={"ID":"74751610-3411-43d1-ace7-1a964f330734","Type":"ContainerStarted","Data":"1f24f1db5e09c3d77f04658ed0aa0684e9dc405505881cb8d20339280e354f3d"} Jul 20 15:59:55.186426 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:55.186380 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" event={"ID":"74751610-3411-43d1-ace7-1a964f330734","Type":"ContainerStarted","Data":"10190fc8f402ed6fc51c2a42edd779e897cb6cba6a7209845a94be696d6b51c7"} Jul 20 15:59:55.204034 ip-10-0-142-55 kubenswrapper[2559]: I0720 15:59:55.203957 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" podStartSLOduration=2.079378518 podStartE2EDuration="3.203939184s" totalImagesPullingTime="1.124560666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:59:55.203097939 +0000 UTC m=+533.381775012" watchObservedRunningTime="2026-07-20 15:59:55.203939184 +0000 UTC m=+533.382616258" Jul 20 16:00:00.135929 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:00.135891 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz"] Jul 20 16:00:00.139411 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:00.139395 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" Jul 20 16:00:00.141336 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:00.141317 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbzb7\"" Jul 20 16:00:00.146066 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:00.146044 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz"] Jul 20 16:00:00.150771 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:00.150753 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" Jul 20 16:00:00.287357 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:00.287198 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz"] Jul 20 16:00:00.289553 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:00:00.289520 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10aba21d_f16a_4903_8086_20556f80ea4d.slice/crio-bd2ef702da5647f840ca5251558bfe8de891cf2078272371bd10b688976401a9 WatchSource:0}: Error finding container bd2ef702da5647f840ca5251558bfe8de891cf2078272371bd10b688976401a9: Status 404 returned error can't find the container with id bd2ef702da5647f840ca5251558bfe8de891cf2078272371bd10b688976401a9 Jul 20 16:00:01.219466 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:01.219435 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" event={"ID":"10aba21d-f16a-4903-8086-20556f80ea4d","Type":"ContainerStarted","Data":"bd2ef702da5647f840ca5251558bfe8de891cf2078272371bd10b688976401a9"} Jul 20 16:00:03.112753 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:03.112717 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 16:00:03.113209 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:03.112782 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 16:00:03.125446 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:03.125423 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 16:00:03.243978 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:03.243946 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd" Jul 20 16:00:06.245829 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:06.245748 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" event={"ID":"10aba21d-f16a-4903-8086-20556f80ea4d","Type":"ContainerStarted","Data":"a7d81dd29cd3f5185217bcb2b3fb6290cb13ac54c1bfe0aa5097dad08d9ebfe2"} Jul 20 16:00:06.261175 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:06.261131 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" podStartSLOduration=0.897175079 podStartE2EDuration="6.261112701s" totalImagesPullingTime="5.363937622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:00:06.259463625 +0000 UTC m=+544.438140702" watchObservedRunningTime="2026-07-20 16:00:06.261112701 +0000 UTC m=+544.439789774" Jul 20 16:00:07.251756 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:07.251719 2559 generic.go:356] "Generic (PLEG): container finished" podID="10aba21d-f16a-4903-8086-20556f80ea4d" containerID="a7d81dd29cd3f5185217bcb2b3fb6290cb13ac54c1bfe0aa5097dad08d9ebfe2" exitCode=0 Jul 20 16:00:07.252134 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:07.251786 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" event={"ID":"10aba21d-f16a-4903-8086-20556f80ea4d","Type":"ContainerDied","Data":"a7d81dd29cd3f5185217bcb2b3fb6290cb13ac54c1bfe0aa5097dad08d9ebfe2"} Jul 20 16:00:08.389963 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:08.389939 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" Jul 20 16:00:08.961998 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:08.961966 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss"] Jul 20 16:00:08.962348 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:08.962336 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10aba21d-f16a-4903-8086-20556f80ea4d" containerName="cleanup" Jul 20 16:00:08.962397 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:08.962350 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10aba21d-f16a-4903-8086-20556f80ea4d" containerName="cleanup" Jul 20 16:00:08.962431 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:08.962424 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="10aba21d-f16a-4903-8086-20556f80ea4d" containerName="cleanup" Jul 20 16:00:08.965469 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:08.965453 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:08.967560 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:08.967534 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 20 16:00:08.974527 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:08.974504 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss"] Jul 20 16:00:09.051301 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.051274 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.051477 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.051316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f165792-1f0b-4c1e-9a8a-715234e03d22-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.051477 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.051345 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.051477 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.051417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.051477 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.051461 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8crd\" (UniqueName: \"kubernetes.io/projected/1f165792-1f0b-4c1e-9a8a-715234e03d22-kube-api-access-q8crd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.051646 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.051531 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.152910 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.152879 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.153148 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.152926 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f165792-1f0b-4c1e-9a8a-715234e03d22-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.153148 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.152953 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.153148 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.152987 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.153148 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.153005 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8crd\" (UniqueName: \"kubernetes.io/projected/1f165792-1f0b-4c1e-9a8a-715234e03d22-kube-api-access-q8crd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.153148 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.153088 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.153434 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.153374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.153434 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.153424 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.153501 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.153448 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.155356 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.155337 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f165792-1f0b-4c1e-9a8a-715234e03d22-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.155517 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.155501 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f165792-1f0b-4c1e-9a8a-715234e03d22-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.160467 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.160447 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8crd\" (UniqueName: \"kubernetes.io/projected/1f165792-1f0b-4c1e-9a8a-715234e03d22-kube-api-access-q8crd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c68ss\" (UID: \"1f165792-1f0b-4c1e-9a8a-715234e03d22\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.262460 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.262374 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" Jul 20 16:00:09.262460 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.262408 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742720-t2xqz" event={"ID":"10aba21d-f16a-4903-8086-20556f80ea4d","Type":"ContainerDied","Data":"bd2ef702da5647f840ca5251558bfe8de891cf2078272371bd10b688976401a9"} Jul 20 16:00:09.262650 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.262459 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd2ef702da5647f840ca5251558bfe8de891cf2078272371bd10b688976401a9" Jul 20 16:00:09.277727 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.277704 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:09.413854 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:09.413833 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss"] Jul 20 16:00:09.416186 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:00:09.416161 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f165792_1f0b_4c1e_9a8a_715234e03d22.slice/crio-ce216b0eef1aa71e4423dd1ff7e2806d859a90bcfc070e86049f9154b2cdbf3b WatchSource:0}: Error finding container ce216b0eef1aa71e4423dd1ff7e2806d859a90bcfc070e86049f9154b2cdbf3b: Status 404 returned error can't find the container with id ce216b0eef1aa71e4423dd1ff7e2806d859a90bcfc070e86049f9154b2cdbf3b Jul 20 16:00:10.268135 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:10.268100 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" event={"ID":"1f165792-1f0b-4c1e-9a8a-715234e03d22","Type":"ContainerStarted","Data":"ce216b0eef1aa71e4423dd1ff7e2806d859a90bcfc070e86049f9154b2cdbf3b"} Jul 20 16:00:44.434104 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:44.434065 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" event={"ID":"1f165792-1f0b-4c1e-9a8a-715234e03d22","Type":"ContainerStarted","Data":"b020f5c4f655c94fbeb8d873c7c3c3bc660f9105b14fffb70e83e8a3c3acf554"} Jul 20 16:00:44.452811 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:44.452769 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" podStartSLOduration=2.048494176 podStartE2EDuration="36.452752951s" totalImagesPullingTime="34.404258775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:00:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:00:44.450030519 +0000 UTC m=+582.628707606" watchObservedRunningTime="2026-07-20 16:00:44.452752951 +0000 UTC m=+582.631430023" Jul 20 16:00:45.281083 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.279410 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c"] Jul 20 16:00:45.285130 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.285108 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.287076 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.287046 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 20 16:00:45.293462 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.293440 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c"] Jul 20 16:00:45.393311 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.393275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.393311 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.393317 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.393521 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.393340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4ncp\" (UniqueName: \"kubernetes.io/projected/4e6890d0-395b-4bac-acf6-478430768357-kube-api-access-f4ncp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.393521 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.393440 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.393521 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.393475 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.393633 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.393590 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6890d0-395b-4bac-acf6-478430768357-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495154 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495116 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495154 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495158 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495627 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495283 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4ncp\" (UniqueName: \"kubernetes.io/projected/4e6890d0-395b-4bac-acf6-478430768357-kube-api-access-f4ncp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495627 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495342 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495627 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495374 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495627 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495474 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6890d0-395b-4bac-acf6-478430768357-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495627 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495475 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495627 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495528 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.495903 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.495829 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.497722 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.497692 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e6890d0-395b-4bac-acf6-478430768357-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.497952 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.497935 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6890d0-395b-4bac-acf6-478430768357-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.502910 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.502893 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4ncp\" (UniqueName: \"kubernetes.io/projected/4e6890d0-395b-4bac-acf6-478430768357-kube-api-access-f4ncp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnl7c\" (UID: \"4e6890d0-395b-4bac-acf6-478430768357\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.596104 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.596072 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:45.733050 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:45.733022 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c"] Jul 20 16:00:45.735065 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:00:45.735031 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6890d0_395b_4bac_acf6_478430768357.slice/crio-a08fc04f145c75feb15ebd3bc6ad91a7a9b41407947995e1d213fdb3f982afeb WatchSource:0}: Error finding container a08fc04f145c75feb15ebd3bc6ad91a7a9b41407947995e1d213fdb3f982afeb: Status 404 returned error can't find the container with id a08fc04f145c75feb15ebd3bc6ad91a7a9b41407947995e1d213fdb3f982afeb Jul 20 16:00:46.444921 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:46.444886 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" event={"ID":"4e6890d0-395b-4bac-acf6-478430768357","Type":"ContainerStarted","Data":"a08fc04f145c75feb15ebd3bc6ad91a7a9b41407947995e1d213fdb3f982afeb"} Jul 20 16:00:48.456001 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:48.455969 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" event={"ID":"4e6890d0-395b-4bac-acf6-478430768357","Type":"ContainerStarted","Data":"4aa54683bed712ee99142efbd0e9ff1ba08e50309641fae6a06c2db703f22896"} Jul 20 16:00:48.476575 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:48.476522 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" podStartSLOduration=1.194226471 podStartE2EDuration="3.476508637s" totalImagesPullingTime="2.282282166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:00:48.474489804 +0000 UTC m=+586.653166876" watchObservedRunningTime="2026-07-20 16:00:48.476508637 +0000 UTC m=+586.655185710" Jul 20 16:00:49.278046 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:49.278009 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:49.278231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:49.278104 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:49.290199 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:49.290169 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:49.471492 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:49.471465 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c68ss" Jul 20 16:00:55.597307 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:55.597276 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:55.597697 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:55.597339 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:55.610360 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:55.610338 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:56.505109 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:56.505081 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnl7c" Jul 20 16:00:57.881653 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:57.881619 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6"] Jul 20 16:00:57.885028 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:57.885012 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:57.886877 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:57.886854 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 20 16:00:57.899829 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:57.899805 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6"] Jul 20 16:00:58.016387 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.016357 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.016564 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.016394 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.016564 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.016462 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.016564 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.016539 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.016701 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.016587 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/211e1746-4644-40fb-8abe-d5e1e10932b3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.016701 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.016616 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4924c\" (UniqueName: \"kubernetes.io/projected/211e1746-4644-40fb-8abe-d5e1e10932b3-kube-api-access-4924c\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118048 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118007 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118062 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118325 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118275 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118377 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118358 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/211e1746-4644-40fb-8abe-d5e1e10932b3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118419 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4924c\" (UniqueName: \"kubernetes.io/projected/211e1746-4644-40fb-8abe-d5e1e10932b3-kube-api-access-4924c\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118534 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118512 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118607 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118543 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.118665 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.118607 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.120418 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.120396 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/211e1746-4644-40fb-8abe-d5e1e10932b3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.120970 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.120949 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/211e1746-4644-40fb-8abe-d5e1e10932b3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.129906 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.129886 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4924c\" (UniqueName: \"kubernetes.io/projected/211e1746-4644-40fb-8abe-d5e1e10932b3-kube-api-access-4924c\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6\" (UID: \"211e1746-4644-40fb-8abe-d5e1e10932b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.194375 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.194290 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:00:58.331927 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.331773 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6"] Jul 20 16:00:58.334421 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:00:58.334398 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod211e1746_4644_40fb_8abe_d5e1e10932b3.slice/crio-c5a9193eb306087c9e89ade2606c7cb8e2d8d4f49eea3cc060dc1b24bce160b2 WatchSource:0}: Error finding container c5a9193eb306087c9e89ade2606c7cb8e2d8d4f49eea3cc060dc1b24bce160b2: Status 404 returned error can't find the container with id c5a9193eb306087c9e89ade2606c7cb8e2d8d4f49eea3cc060dc1b24bce160b2 Jul 20 16:00:58.504974 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:00:58.504900 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" event={"ID":"211e1746-4644-40fb-8abe-d5e1e10932b3","Type":"ContainerStarted","Data":"c5a9193eb306087c9e89ade2606c7cb8e2d8d4f49eea3cc060dc1b24bce160b2"} Jul 20 16:01:42.730007 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:01:42.729966 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" event={"ID":"211e1746-4644-40fb-8abe-d5e1e10932b3","Type":"ContainerStarted","Data":"35a9b88e02a1de09fbc4bb5d95fc5f9cca25c447478519ef2db6e52322107a47"} Jul 20 16:01:42.756129 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:01:42.756083 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" podStartSLOduration=1.5016267349999999 podStartE2EDuration="45.756067853s" totalImagesPullingTime="44.254441118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:01:42.753580766 +0000 UTC m=+640.932257839" watchObservedRunningTime="2026-07-20 16:01:42.756067853 +0000 UTC m=+640.934744928" Jul 20 16:01:48.194765 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:01:48.194727 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:01:48.197335 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:01:48.194878 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:01:48.207730 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:01:48.207708 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:01:48.768817 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:01:48.768787 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6" Jul 20 16:13:14.572169 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.572135 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk"] Jul 20 16:13:14.575933 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.575916 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.578368 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.578350 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-5d2e68d4-kserve-self-signed-certs\"" Jul 20 16:13:14.599411 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.599379 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk"] Jul 20 16:13:14.658172 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.658133 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tmp-dir\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.658172 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.658174 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tls-certs\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.658434 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.658269 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-dshm\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.658434 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.658340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f449\" (UniqueName: \"kubernetes.io/projected/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-kube-api-access-7f449\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.658434 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.658369 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-model-cache\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.658563 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.658447 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-home\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.759890 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.759853 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f449\" (UniqueName: \"kubernetes.io/projected/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-kube-api-access-7f449\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.759890 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.759892 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-model-cache\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.760353 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.760015 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-home\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.760353 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.760055 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tmp-dir\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.760353 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.760073 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tls-certs\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.760353 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.760136 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-dshm\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.760353 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.760223 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-model-cache\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.760624 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.760374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-home\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.760624 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.760381 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tmp-dir\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.762385 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.762364 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-dshm\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.762718 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.762702 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tls-certs\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.768586 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.768565 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f449\" (UniqueName: \"kubernetes.io/projected/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-kube-api-access-7f449\") pod \"e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:14.885240 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:14.885153 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:15.030345 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:15.030316 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk"] Jul 20 16:13:15.033002 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:13:15.032978 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod168ff7b3_9ff4_4464_a88d_b99a19d3f01e.slice/crio-9ed87bb28a60bf0ac9c0bcac39002fd649011331156c447ff2dbd9cb4ccaaa87 WatchSource:0}: Error finding container 9ed87bb28a60bf0ac9c0bcac39002fd649011331156c447ff2dbd9cb4ccaaa87: Status 404 returned error can't find the container with id 9ed87bb28a60bf0ac9c0bcac39002fd649011331156c447ff2dbd9cb4ccaaa87 Jul 20 16:13:15.146204 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:15.146115 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" event={"ID":"168ff7b3-9ff4-4464-a88d-b99a19d3f01e","Type":"ContainerStarted","Data":"950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c"} Jul 20 16:13:15.146204 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:15.146157 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" event={"ID":"168ff7b3-9ff4-4464-a88d-b99a19d3f01e","Type":"ContainerStarted","Data":"9ed87bb28a60bf0ac9c0bcac39002fd649011331156c447ff2dbd9cb4ccaaa87"} Jul 20 16:13:15.163564 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:15.163518 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" podStartSLOduration=1.163499349 podStartE2EDuration="1.163499349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:13:15.161753378 +0000 UTC m=+1333.340430448" watchObservedRunningTime="2026-07-20 16:13:15.163499349 +0000 UTC m=+1333.342176423" Jul 20 16:13:24.885887 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:24.885802 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:24.885887 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:24.885852 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:24.898740 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:24.898715 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:25.206956 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:25.206871 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:13:51.639844 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:13:51.639814 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk"] Jul 20 16:14:06.660343 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.660282 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" podUID="168ff7b3-9ff4-4464-a88d-b99a19d3f01e" containerName="main" containerID="cri-o://950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c" gracePeriod=45 Jul 20 16:14:06.902018 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.901996 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:14:06.959079 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959013 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tls-certs\") pod \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " Jul 20 16:14:06.959079 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959068 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-home\") pod \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " Jul 20 16:14:06.959273 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959099 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-dshm\") pod \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " Jul 20 16:14:06.959273 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959126 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f449\" (UniqueName: \"kubernetes.io/projected/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-kube-api-access-7f449\") pod \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " Jul 20 16:14:06.959273 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959185 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-model-cache\") pod \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " Jul 20 16:14:06.959273 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959210 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tmp-dir\") pod \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\" (UID: \"168ff7b3-9ff4-4464-a88d-b99a19d3f01e\") " Jul 20 16:14:06.959483 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959344 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-home" (OuterVolumeSpecName: "home") pod "168ff7b3-9ff4-4464-a88d-b99a19d3f01e" (UID: "168ff7b3-9ff4-4464-a88d-b99a19d3f01e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:14:06.959483 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959473 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-model-cache" (OuterVolumeSpecName: "model-cache") pod "168ff7b3-9ff4-4464-a88d-b99a19d3f01e" (UID: "168ff7b3-9ff4-4464-a88d-b99a19d3f01e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:14:06.959596 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959511 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-home\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:14:06.959652 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.959598 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "168ff7b3-9ff4-4464-a88d-b99a19d3f01e" (UID: "168ff7b3-9ff4-4464-a88d-b99a19d3f01e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:14:06.961520 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.961495 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "168ff7b3-9ff4-4464-a88d-b99a19d3f01e" (UID: "168ff7b3-9ff4-4464-a88d-b99a19d3f01e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:14:06.961617 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.961544 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-dshm" (OuterVolumeSpecName: "dshm") pod "168ff7b3-9ff4-4464-a88d-b99a19d3f01e" (UID: "168ff7b3-9ff4-4464-a88d-b99a19d3f01e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:14:06.961617 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:06.961586 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-kube-api-access-7f449" (OuterVolumeSpecName: "kube-api-access-7f449") pod "168ff7b3-9ff4-4464-a88d-b99a19d3f01e" (UID: "168ff7b3-9ff4-4464-a88d-b99a19d3f01e"). InnerVolumeSpecName "kube-api-access-7f449". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:14:07.060553 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.060521 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tls-certs\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:14:07.060553 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.060547 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-dshm\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:14:07.060553 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.060556 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f449\" (UniqueName: \"kubernetes.io/projected/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-kube-api-access-7f449\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:14:07.060830 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.060566 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-model-cache\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:14:07.060830 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.060575 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/168ff7b3-9ff4-4464-a88d-b99a19d3f01e-tmp-dir\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:14:07.400976 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.400948 2559 generic.go:356] "Generic (PLEG): container finished" podID="168ff7b3-9ff4-4464-a88d-b99a19d3f01e" containerID="950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c" exitCode=0 Jul 20 16:14:07.401164 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.401009 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" Jul 20 16:14:07.401164 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.401012 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" event={"ID":"168ff7b3-9ff4-4464-a88d-b99a19d3f01e","Type":"ContainerDied","Data":"950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c"} Jul 20 16:14:07.401164 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.401103 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk" event={"ID":"168ff7b3-9ff4-4464-a88d-b99a19d3f01e","Type":"ContainerDied","Data":"9ed87bb28a60bf0ac9c0bcac39002fd649011331156c447ff2dbd9cb4ccaaa87"} Jul 20 16:14:07.401164 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.401122 2559 scope.go:117] "RemoveContainer" containerID="950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c" Jul 20 16:14:07.410522 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.410504 2559 scope.go:117] "RemoveContainer" containerID="950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c" Jul 20 16:14:07.410821 ip-10-0-142-55 kubenswrapper[2559]: E0720 16:14:07.410801 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c\": container with ID starting with 950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c not found: ID does not exist" containerID="950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c" Jul 20 16:14:07.410887 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.410829 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c"} err="failed to get container status \"950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c\": rpc error: code = NotFound desc = could not find container \"950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c\": container with ID starting with 950174d861aad53deb1b3e5393be5324c6fa02e9856f61ccfaad08ef72a41b0c not found: ID does not exist" Jul 20 16:14:07.421653 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.421627 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk"] Jul 20 16:14:07.427038 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:07.427018 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-5d2e68d4-kserve-59886bf4f5-49xjk"] Jul 20 16:14:08.317971 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:14:08.317942 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="168ff7b3-9ff4-4464-a88d-b99a19d3f01e" path="/var/lib/kubelet/pods/168ff7b3-9ff4-4464-a88d-b99a19d3f01e/volumes" Jul 20 16:23:33.761915 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.761876 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c"] Jul 20 16:23:33.762323 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.762313 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168ff7b3-9ff4-4464-a88d-b99a19d3f01e" containerName="main" Jul 20 16:23:33.762364 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.762325 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="168ff7b3-9ff4-4464-a88d-b99a19d3f01e" containerName="main" Jul 20 16:23:33.762414 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.762403 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="168ff7b3-9ff4-4464-a88d-b99a19d3f01e" containerName="main" Jul 20 16:23:33.765568 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.765547 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.768090 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.768067 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-aef3b1b4\"/\"default-dockercfg-m2t5w\"" Jul 20 16:23:33.768090 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.768081 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-aef3b1b4\"/\"openshift-service-ca.crt\"" Jul 20 16:23:33.768267 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.768075 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-aef3b1b4\"/\"kube-root-ca.crt\"" Jul 20 16:23:33.768267 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.768075 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-aef3b1b4\"/\"e2e-lifecycle-model-aef3b1b4-kserve-self-signed-certs\"" Jul 20 16:23:33.775841 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.775813 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c"] Jul 20 16:23:33.813687 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.813647 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-home\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.813863 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.813723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-dshm\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.813863 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.813756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wmwn\" (UniqueName: \"kubernetes.io/projected/48751ade-d667-489d-acf0-0aeb815ce25b-kube-api-access-8wmwn\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.813941 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.813890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-tmp-dir\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.813941 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.813927 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-model-cache\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.814007 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.813953 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48751ade-d667-489d-acf0-0aeb815ce25b-tls-certs\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.914564 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.914524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-home\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.914564 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.914566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-dshm\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.914862 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.914592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wmwn\" (UniqueName: \"kubernetes.io/projected/48751ade-d667-489d-acf0-0aeb815ce25b-kube-api-access-8wmwn\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.914862 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.914656 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-tmp-dir\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.914862 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.914712 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-model-cache\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.914862 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.914745 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48751ade-d667-489d-acf0-0aeb815ce25b-tls-certs\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.915069 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.914981 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-home\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.915069 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.915029 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-tmp-dir\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.915138 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.915084 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-model-cache\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.916965 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.916946 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-dshm\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.917358 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.917337 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48751ade-d667-489d-acf0-0aeb815ce25b-tls-certs\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:33.922945 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:33.922916 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wmwn\" (UniqueName: \"kubernetes.io/projected/48751ade-d667-489d-acf0-0aeb815ce25b-kube-api-access-8wmwn\") pod \"e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:34.076609 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:34.076570 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:34.211590 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:34.211513 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c"] Jul 20 16:23:34.214221 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:23:34.214190 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48751ade_d667_489d_acf0_0aeb815ce25b.slice/crio-e8337c89fe85b0a4e833a91e3f58e4b92f4743a6b871ca8857de9d8ccf4fb3e8 WatchSource:0}: Error finding container e8337c89fe85b0a4e833a91e3f58e4b92f4743a6b871ca8857de9d8ccf4fb3e8: Status 404 returned error can't find the container with id e8337c89fe85b0a4e833a91e3f58e4b92f4743a6b871ca8857de9d8ccf4fb3e8 Jul 20 16:23:35.206158 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:35.206122 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" event={"ID":"48751ade-d667-489d-acf0-0aeb815ce25b","Type":"ContainerStarted","Data":"e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32"} Jul 20 16:23:35.206641 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:35.206168 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" event={"ID":"48751ade-d667-489d-acf0-0aeb815ce25b","Type":"ContainerStarted","Data":"e8337c89fe85b0a4e833a91e3f58e4b92f4743a6b871ca8857de9d8ccf4fb3e8"} Jul 20 16:23:35.223774 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:35.223721 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" podStartSLOduration=2.223702799 podStartE2EDuration="2.223702799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:23:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:23:35.221776234 +0000 UTC m=+1953.400453306" watchObservedRunningTime="2026-07-20 16:23:35.223702799 +0000 UTC m=+1953.402379863" Jul 20 16:23:44.077586 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:44.077536 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:44.077586 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:44.077601 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:44.090495 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:44.090470 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:23:44.261728 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:23:44.261667 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:24:31.576234 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:31.576195 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c"] Jul 20 16:24:46.597280 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.597220 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" podUID="48751ade-d667-489d-acf0-0aeb815ce25b" containerName="main" containerID="cri-o://e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32" gracePeriod=45 Jul 20 16:24:46.847101 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.847079 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:24:46.923612 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.923543 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wmwn\" (UniqueName: \"kubernetes.io/projected/48751ade-d667-489d-acf0-0aeb815ce25b-kube-api-access-8wmwn\") pod \"48751ade-d667-489d-acf0-0aeb815ce25b\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " Jul 20 16:24:46.923612 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.923591 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-dshm\") pod \"48751ade-d667-489d-acf0-0aeb815ce25b\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " Jul 20 16:24:46.923830 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.923638 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48751ade-d667-489d-acf0-0aeb815ce25b-tls-certs\") pod \"48751ade-d667-489d-acf0-0aeb815ce25b\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " Jul 20 16:24:46.923830 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.923707 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-tmp-dir\") pod \"48751ade-d667-489d-acf0-0aeb815ce25b\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " Jul 20 16:24:46.923830 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.923730 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-model-cache\") pod \"48751ade-d667-489d-acf0-0aeb815ce25b\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " Jul 20 16:24:46.923830 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.923777 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-home\") pod \"48751ade-d667-489d-acf0-0aeb815ce25b\" (UID: \"48751ade-d667-489d-acf0-0aeb815ce25b\") " Jul 20 16:24:46.924067 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.924037 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-model-cache" (OuterVolumeSpecName: "model-cache") pod "48751ade-d667-489d-acf0-0aeb815ce25b" (UID: "48751ade-d667-489d-acf0-0aeb815ce25b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:24:46.924132 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.924021 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48751ade-d667-489d-acf0-0aeb815ce25b" (UID: "48751ade-d667-489d-acf0-0aeb815ce25b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:24:46.924132 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.924063 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-home" (OuterVolumeSpecName: "home") pod "48751ade-d667-489d-acf0-0aeb815ce25b" (UID: "48751ade-d667-489d-acf0-0aeb815ce25b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:24:46.924222 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.924145 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-tmp-dir\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:24:46.924222 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.924164 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-model-cache\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:24:46.926012 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.925988 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48751ade-d667-489d-acf0-0aeb815ce25b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48751ade-d667-489d-acf0-0aeb815ce25b" (UID: "48751ade-d667-489d-acf0-0aeb815ce25b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:24:46.926207 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.926192 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48751ade-d667-489d-acf0-0aeb815ce25b-kube-api-access-8wmwn" (OuterVolumeSpecName: "kube-api-access-8wmwn") pod "48751ade-d667-489d-acf0-0aeb815ce25b" (UID: "48751ade-d667-489d-acf0-0aeb815ce25b"). InnerVolumeSpecName "kube-api-access-8wmwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:24:46.926207 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:46.926196 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-dshm" (OuterVolumeSpecName: "dshm") pod "48751ade-d667-489d-acf0-0aeb815ce25b" (UID: "48751ade-d667-489d-acf0-0aeb815ce25b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:24:47.025001 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.024967 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-home\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:24:47.025001 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.024994 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wmwn\" (UniqueName: \"kubernetes.io/projected/48751ade-d667-489d-acf0-0aeb815ce25b-kube-api-access-8wmwn\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:24:47.025001 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.025004 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48751ade-d667-489d-acf0-0aeb815ce25b-dshm\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:24:47.025215 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.025013 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48751ade-d667-489d-acf0-0aeb815ce25b-tls-certs\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:24:47.576096 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.576058 2559 generic.go:356] "Generic (PLEG): container finished" podID="48751ade-d667-489d-acf0-0aeb815ce25b" containerID="e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32" exitCode=0 Jul 20 16:24:47.576285 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.576108 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" Jul 20 16:24:47.576285 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.576126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" event={"ID":"48751ade-d667-489d-acf0-0aeb815ce25b","Type":"ContainerDied","Data":"e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32"} Jul 20 16:24:47.576285 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.576165 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c" event={"ID":"48751ade-d667-489d-acf0-0aeb815ce25b","Type":"ContainerDied","Data":"e8337c89fe85b0a4e833a91e3f58e4b92f4743a6b871ca8857de9d8ccf4fb3e8"} Jul 20 16:24:47.576285 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.576192 2559 scope.go:117] "RemoveContainer" containerID="e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32" Jul 20 16:24:47.586400 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.586379 2559 scope.go:117] "RemoveContainer" containerID="e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32" Jul 20 16:24:47.586663 ip-10-0-142-55 kubenswrapper[2559]: E0720 16:24:47.586643 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32\": container with ID starting with e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32 not found: ID does not exist" containerID="e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32" Jul 20 16:24:47.586757 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.586692 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32"} err="failed to get container status \"e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32\": rpc error: code = NotFound desc = could not find container \"e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32\": container with ID starting with e25a355d01db7a3dc5d00280bd7261f53e039b15a4fa4a90930399f2ba3fac32 not found: ID does not exist" Jul 20 16:24:47.597784 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.597755 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c"] Jul 20 16:24:47.602717 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:47.602693 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-aef3b1b4/e2e-lifecycle-model-aef3b1b4-kserve-796c8f646c-9lx7c"] Jul 20 16:24:48.318455 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:24:48.318420 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48751ade-d667-489d-acf0-0aeb815ce25b" path="/var/lib/kubelet/pods/48751ade-d667-489d-acf0-0aeb815ce25b/volumes" Jul 20 16:26:37.674127 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.674093 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f"] Jul 20 16:26:37.674558 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.674521 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48751ade-d667-489d-acf0-0aeb815ce25b" containerName="main" Jul 20 16:26:37.674558 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.674531 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="48751ade-d667-489d-acf0-0aeb815ce25b" containerName="main" Jul 20 16:26:37.674640 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.674598 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="48751ade-d667-489d-acf0-0aeb815ce25b" containerName="main" Jul 20 16:26:37.677593 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.677578 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.680796 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.680773 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f3d110\"/\"test-model-f3d110-kserve-self-signed-certs\"" Jul 20 16:26:37.681166 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.681147 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f3d110\"/\"kube-root-ca.crt\"" Jul 20 16:26:37.681215 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.681174 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f3d110\"/\"default-dockercfg-zf76l\"" Jul 20 16:26:37.681340 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.681315 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f3d110\"/\"openshift-service-ca.crt\"" Jul 20 16:26:37.690144 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.690121 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f"] Jul 20 16:26:37.713339 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.713305 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdzb9\" (UniqueName: \"kubernetes.io/projected/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-kube-api-access-xdzb9\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.713496 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.713368 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tmp-dir\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.713496 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.713425 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-model-cache\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.713496 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.713439 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-dshm\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.713496 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.713472 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tls-certs\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.713747 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.713590 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-home\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814068 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814041 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tls-certs\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814238 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814101 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-home\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814238 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814128 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdzb9\" (UniqueName: \"kubernetes.io/projected/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-kube-api-access-xdzb9\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814238 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814169 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tmp-dir\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814238 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814196 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-model-cache\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814238 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814210 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-dshm\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814585 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814542 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-home\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814652 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814602 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tmp-dir\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.814652 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.814631 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-model-cache\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.816541 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.816523 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-dshm\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.816844 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.816824 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tls-certs\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.822995 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.822972 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdzb9\" (UniqueName: \"kubernetes.io/projected/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-kube-api-access-xdzb9\") pod \"test-model-f3d110-kserve-567dd99d99-wkc2f\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:37.987265 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:37.987165 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:38.128238 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:38.128214 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f"] Jul 20 16:26:38.129959 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:26:38.129928 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b3e0f22_e12d_4f82_b7f2_a6127cd386e3.slice/crio-d06b56e9a2664036dbee88b3d2590ba7681186128ed37ed30c43a84b96425191 WatchSource:0}: Error finding container d06b56e9a2664036dbee88b3d2590ba7681186128ed37ed30c43a84b96425191: Status 404 returned error can't find the container with id d06b56e9a2664036dbee88b3d2590ba7681186128ed37ed30c43a84b96425191 Jul 20 16:26:39.137379 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:39.137342 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" event={"ID":"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3","Type":"ContainerStarted","Data":"a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0"} Jul 20 16:26:39.137877 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:39.137388 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" event={"ID":"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3","Type":"ContainerStarted","Data":"d06b56e9a2664036dbee88b3d2590ba7681186128ed37ed30c43a84b96425191"} Jul 20 16:26:39.158909 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:39.158863 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" podStartSLOduration=2.158847266 podStartE2EDuration="2.158847266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:26:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:26:39.156773833 +0000 UTC m=+2137.335450906" watchObservedRunningTime="2026-07-20 16:26:39.158847266 +0000 UTC m=+2137.337524339" Jul 20 16:26:47.987935 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:47.987899 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:47.988337 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:47.987974 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:48.000400 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:48.000373 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:26:48.195374 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:26:48.195348 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:27:32.780083 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.780048 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf"] Jul 20 16:27:32.783745 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.783721 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:32.787028 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.787006 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-18d8b0\"/\"openshift-service-ca.crt\"" Jul 20 16:27:32.787375 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.787354 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-18d8b0\"/\"kube-root-ca.crt\"" Jul 20 16:27:32.787900 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.787882 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-18d8b0\"/\"default-dockercfg-zlr8f\"" Jul 20 16:27:32.788133 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.788119 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-18d8b0\"/\"test-model-18d8b0-kserve-self-signed-certs\"" Jul 20 16:27:32.801982 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.801953 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf"] Jul 20 16:27:32.948737 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.948697 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w24rn\" (UniqueName: \"kubernetes.io/projected/a06fd033-a909-40b6-be0c-ba6a958287c5-kube-api-access-w24rn\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:32.948932 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.948747 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-dshm\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:32.948932 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.948801 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a06fd033-a909-40b6-be0c-ba6a958287c5-tls-certs\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:32.948932 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.948827 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-model-cache\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:32.948932 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.948898 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-home\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:32.949105 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:32.948944 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-tmp-dir\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.049770 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.049731 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-model-cache\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.049955 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.049825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-home\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.049955 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.049875 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-tmp-dir\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.049955 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.049935 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w24rn\" (UniqueName: \"kubernetes.io/projected/a06fd033-a909-40b6-be0c-ba6a958287c5-kube-api-access-w24rn\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.050127 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.049958 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-dshm\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.050127 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.050017 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a06fd033-a909-40b6-be0c-ba6a958287c5-tls-certs\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.050127 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.050100 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-model-cache\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.050308 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.050195 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-home\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.050308 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.050295 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-tmp-dir\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.052415 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.052389 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-dshm\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.052558 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.052543 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a06fd033-a909-40b6-be0c-ba6a958287c5-tls-certs\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.064440 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.064417 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w24rn\" (UniqueName: \"kubernetes.io/projected/a06fd033-a909-40b6-be0c-ba6a958287c5-kube-api-access-w24rn\") pod \"test-model-18d8b0-kserve-57b598c675-mjljf\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.093316 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.093285 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:33.235035 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.235003 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf"] Jul 20 16:27:33.237529 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:27:33.237496 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda06fd033_a909_40b6_be0c_ba6a958287c5.slice/crio-59e236d2f82259d53093275a337ecad2813c464a67ce5fb0448e98b503205bc9 WatchSource:0}: Error finding container 59e236d2f82259d53093275a337ecad2813c464a67ce5fb0448e98b503205bc9: Status 404 returned error can't find the container with id 59e236d2f82259d53093275a337ecad2813c464a67ce5fb0448e98b503205bc9 Jul 20 16:27:33.403807 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.403713 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" event={"ID":"a06fd033-a909-40b6-be0c-ba6a958287c5","Type":"ContainerStarted","Data":"79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684"} Jul 20 16:27:33.403807 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.403765 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" event={"ID":"a06fd033-a909-40b6-be0c-ba6a958287c5","Type":"ContainerStarted","Data":"59e236d2f82259d53093275a337ecad2813c464a67ce5fb0448e98b503205bc9"} Jul 20 16:27:33.421081 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:33.421027 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" podStartSLOduration=1.42100918 podStartE2EDuration="1.42100918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:27:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:27:33.420207055 +0000 UTC m=+2191.598884129" watchObservedRunningTime="2026-07-20 16:27:33.42100918 +0000 UTC m=+2191.599686265" Jul 20 16:27:43.093650 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:43.093614 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:43.094173 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:43.093668 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:43.106259 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:43.106235 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:27:43.468629 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:27:43.468555 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:28:18.093626 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:18.093593 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f"] Jul 20 16:28:19.133915 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:19.133877 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf"] Jul 20 16:28:33.115931 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.115871 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" podUID="3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" containerName="main" containerID="cri-o://a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0" gracePeriod=45 Jul 20 16:28:33.358254 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.358232 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:28:33.435812 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.435741 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-dshm\") pod \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " Jul 20 16:28:33.435812 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.435797 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-model-cache\") pod \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " Jul 20 16:28:33.436008 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.435847 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tls-certs\") pod \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " Jul 20 16:28:33.436008 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.435866 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdzb9\" (UniqueName: \"kubernetes.io/projected/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-kube-api-access-xdzb9\") pod \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " Jul 20 16:28:33.436008 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.435901 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-home\") pod \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " Jul 20 16:28:33.436008 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.435929 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tmp-dir\") pod \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\" (UID: \"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3\") " Jul 20 16:28:33.436231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.436128 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" (UID: "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:28:33.436231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.436144 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-home" (OuterVolumeSpecName: "home") pod "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" (UID: "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:28:33.436343 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.436229 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" (UID: "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:28:33.436405 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.436355 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-home\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:33.436405 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.436374 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tmp-dir\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:33.436405 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.436390 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-model-cache\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:33.438192 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.438168 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-kube-api-access-xdzb9" (OuterVolumeSpecName: "kube-api-access-xdzb9") pod "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" (UID: "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3"). InnerVolumeSpecName "kube-api-access-xdzb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:28:33.438298 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.438187 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-dshm" (OuterVolumeSpecName: "dshm") pod "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" (UID: "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:28:33.438298 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.438242 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" (UID: "3b3e0f22-e12d-4f82-b7f2-a6127cd386e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:28:33.537421 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.537387 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-dshm\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:33.537421 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.537416 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-tls-certs\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:33.537594 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.537431 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdzb9\" (UniqueName: \"kubernetes.io/projected/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3-kube-api-access-xdzb9\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:33.706162 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.706073 2559 generic.go:356] "Generic (PLEG): container finished" podID="3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" containerID="a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0" exitCode=0 Jul 20 16:28:33.706162 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.706117 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" Jul 20 16:28:33.706162 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.706136 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" event={"ID":"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3","Type":"ContainerDied","Data":"a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0"} Jul 20 16:28:33.706392 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.706177 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f" event={"ID":"3b3e0f22-e12d-4f82-b7f2-a6127cd386e3","Type":"ContainerDied","Data":"d06b56e9a2664036dbee88b3d2590ba7681186128ed37ed30c43a84b96425191"} Jul 20 16:28:33.706392 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.706198 2559 scope.go:117] "RemoveContainer" containerID="a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0" Jul 20 16:28:33.715749 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.715728 2559 scope.go:117] "RemoveContainer" containerID="a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0" Jul 20 16:28:33.716044 ip-10-0-142-55 kubenswrapper[2559]: E0720 16:28:33.716025 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0\": container with ID starting with a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0 not found: ID does not exist" containerID="a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0" Jul 20 16:28:33.716136 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.716051 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0"} err="failed to get container status \"a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0\": rpc error: code = NotFound desc = could not find container \"a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0\": container with ID starting with a022a1bf13d50e741c6821b75296330c65fb4474c89e42c9c4bc818990fbbbf0 not found: ID does not exist" Jul 20 16:28:33.726605 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.726580 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f"] Jul 20 16:28:33.730971 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:33.730950 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-f3d110/test-model-f3d110-kserve-567dd99d99-wkc2f"] Jul 20 16:28:34.154820 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.154773 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" podUID="a06fd033-a909-40b6-be0c-ba6a958287c5" containerName="main" containerID="cri-o://79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684" gracePeriod=45 Jul 20 16:28:34.319318 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.319285 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" path="/var/lib/kubelet/pods/3b3e0f22-e12d-4f82-b7f2-a6127cd386e3/volumes" Jul 20 16:28:34.408847 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.408789 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:28:34.447842 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.447818 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-model-cache\") pod \"a06fd033-a909-40b6-be0c-ba6a958287c5\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " Jul 20 16:28:34.447983 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.447865 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-dshm\") pod \"a06fd033-a909-40b6-be0c-ba6a958287c5\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " Jul 20 16:28:34.447983 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.447910 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-home\") pod \"a06fd033-a909-40b6-be0c-ba6a958287c5\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " Jul 20 16:28:34.447983 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.447936 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-tmp-dir\") pod \"a06fd033-a909-40b6-be0c-ba6a958287c5\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " Jul 20 16:28:34.448162 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.448010 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a06fd033-a909-40b6-be0c-ba6a958287c5-tls-certs\") pod \"a06fd033-a909-40b6-be0c-ba6a958287c5\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " Jul 20 16:28:34.448162 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.448074 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "a06fd033-a909-40b6-be0c-ba6a958287c5" (UID: "a06fd033-a909-40b6-be0c-ba6a958287c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:28:34.448162 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.448113 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w24rn\" (UniqueName: \"kubernetes.io/projected/a06fd033-a909-40b6-be0c-ba6a958287c5-kube-api-access-w24rn\") pod \"a06fd033-a909-40b6-be0c-ba6a958287c5\" (UID: \"a06fd033-a909-40b6-be0c-ba6a958287c5\") " Jul 20 16:28:34.448316 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.448280 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a06fd033-a909-40b6-be0c-ba6a958287c5" (UID: "a06fd033-a909-40b6-be0c-ba6a958287c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:28:34.448316 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.448291 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-home" (OuterVolumeSpecName: "home") pod "a06fd033-a909-40b6-be0c-ba6a958287c5" (UID: "a06fd033-a909-40b6-be0c-ba6a958287c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:28:34.448488 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.448468 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-home\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:34.448546 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.448494 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-tmp-dir\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:34.448546 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.448508 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-model-cache\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:34.450281 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.450253 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-dshm" (OuterVolumeSpecName: "dshm") pod "a06fd033-a909-40b6-be0c-ba6a958287c5" (UID: "a06fd033-a909-40b6-be0c-ba6a958287c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:28:34.450377 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.450353 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a06fd033-a909-40b6-be0c-ba6a958287c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a06fd033-a909-40b6-be0c-ba6a958287c5" (UID: "a06fd033-a909-40b6-be0c-ba6a958287c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:28:34.450494 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.450474 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06fd033-a909-40b6-be0c-ba6a958287c5-kube-api-access-w24rn" (OuterVolumeSpecName: "kube-api-access-w24rn") pod "a06fd033-a909-40b6-be0c-ba6a958287c5" (UID: "a06fd033-a909-40b6-be0c-ba6a958287c5"). InnerVolumeSpecName "kube-api-access-w24rn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:28:34.549664 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.549624 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a06fd033-a909-40b6-be0c-ba6a958287c5-tls-certs\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:34.549664 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.549660 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w24rn\" (UniqueName: \"kubernetes.io/projected/a06fd033-a909-40b6-be0c-ba6a958287c5-kube-api-access-w24rn\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:34.549664 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.549669 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a06fd033-a909-40b6-be0c-ba6a958287c5-dshm\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:28:34.712739 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.712636 2559 generic.go:356] "Generic (PLEG): container finished" podID="a06fd033-a909-40b6-be0c-ba6a958287c5" containerID="79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684" exitCode=0 Jul 20 16:28:34.712739 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.712701 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" Jul 20 16:28:34.712940 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.712732 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" event={"ID":"a06fd033-a909-40b6-be0c-ba6a958287c5","Type":"ContainerDied","Data":"79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684"} Jul 20 16:28:34.712940 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.712777 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf" event={"ID":"a06fd033-a909-40b6-be0c-ba6a958287c5","Type":"ContainerDied","Data":"59e236d2f82259d53093275a337ecad2813c464a67ce5fb0448e98b503205bc9"} Jul 20 16:28:34.712940 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.712799 2559 scope.go:117] "RemoveContainer" containerID="79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684" Jul 20 16:28:34.722623 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.722605 2559 scope.go:117] "RemoveContainer" containerID="79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684" Jul 20 16:28:34.722905 ip-10-0-142-55 kubenswrapper[2559]: E0720 16:28:34.722887 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684\": container with ID starting with 79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684 not found: ID does not exist" containerID="79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684" Jul 20 16:28:34.722962 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.722913 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684"} err="failed to get container status \"79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684\": rpc error: code = NotFound desc = could not find container \"79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684\": container with ID starting with 79ce25b92983bc92bc7ff96ca5b2a8f837ca4586cc23cf2d064ca359c4120684 not found: ID does not exist" Jul 20 16:28:34.736992 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.736970 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf"] Jul 20 16:28:34.742316 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:34.742295 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-18d8b0/test-model-18d8b0-kserve-57b598c675-mjljf"] Jul 20 16:28:36.318440 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:28:36.318408 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a06fd033-a909-40b6-be0c-ba6a958287c5" path="/var/lib/kubelet/pods/a06fd033-a909-40b6-be0c-ba6a958287c5/volumes" Jul 20 16:30:48.777541 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.777509 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb"] Jul 20 16:30:48.778149 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.778123 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06fd033-a909-40b6-be0c-ba6a958287c5" containerName="main" Jul 20 16:30:48.778149 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.778143 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06fd033-a909-40b6-be0c-ba6a958287c5" containerName="main" Jul 20 16:30:48.778286 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.778174 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" containerName="main" Jul 20 16:30:48.778286 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.778183 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" containerName="main" Jul 20 16:30:48.778286 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.778284 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b3e0f22-e12d-4f82-b7f2-a6127cd386e3" containerName="main" Jul 20 16:30:48.778466 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.778302 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a06fd033-a909-40b6-be0c-ba6a958287c5" containerName="main" Jul 20 16:30:48.781490 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.781470 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.783914 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.783890 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-329d3d\"/\"kube-root-ca.crt\"" Jul 20 16:30:48.784038 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.783948 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-329d3d\"/\"auth-test-model-329d3d-kserve-self-signed-certs\"" Jul 20 16:30:48.784038 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.783891 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-329d3d\"/\"default-dockercfg-9svtv\"" Jul 20 16:30:48.784038 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.784022 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-329d3d\"/\"openshift-service-ca.crt\"" Jul 20 16:30:48.791550 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.791528 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb"] Jul 20 16:30:48.862626 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.862587 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q86t2\" (UniqueName: \"kubernetes.io/projected/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-kube-api-access-q86t2\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.862835 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.862642 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tmp-dir\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.862835 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.862753 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-dshm\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.862835 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.862783 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-model-cache\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.862835 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.862812 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-home\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.862985 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.862892 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tls-certs\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964161 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964136 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tls-certs\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964277 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964192 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q86t2\" (UniqueName: \"kubernetes.io/projected/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-kube-api-access-q86t2\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964277 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964219 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tmp-dir\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964277 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964249 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-dshm\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964277 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964266 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-model-cache\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964486 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964330 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-home\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964658 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964637 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tmp-dir\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964775 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964714 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-model-cache\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.964775 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.964765 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-home\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.966640 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.966623 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-dshm\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.967052 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.967032 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tls-certs\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:48.971822 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:48.971801 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q86t2\" (UniqueName: \"kubernetes.io/projected/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-kube-api-access-q86t2\") pod \"auth-test-model-329d3d-kserve-77ff778db7-4ktqb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:49.090953 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:49.090928 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:49.223847 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:49.223820 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb"] Jul 20 16:30:49.225917 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:30:49.225891 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5f79f37_6f6e_4c3f_adc9_4cf1d306a9cb.slice/crio-28c1289cd5f24a9c51102924c7e3fc30ab47fd5631fedee863a6a8d22470ed09 WatchSource:0}: Error finding container 28c1289cd5f24a9c51102924c7e3fc30ab47fd5631fedee863a6a8d22470ed09: Status 404 returned error can't find the container with id 28c1289cd5f24a9c51102924c7e3fc30ab47fd5631fedee863a6a8d22470ed09 Jul 20 16:30:49.373864 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:49.373784 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" event={"ID":"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb","Type":"ContainerStarted","Data":"823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8"} Jul 20 16:30:49.373864 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:49.373821 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" event={"ID":"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb","Type":"ContainerStarted","Data":"28c1289cd5f24a9c51102924c7e3fc30ab47fd5631fedee863a6a8d22470ed09"} Jul 20 16:30:49.392140 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:49.392094 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" podStartSLOduration=1.392079383 podStartE2EDuration="1.392079383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:49.39007983 +0000 UTC m=+2387.568756903" watchObservedRunningTime="2026-07-20 16:30:49.392079383 +0000 UTC m=+2387.570756456" Jul 20 16:30:59.091983 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:59.091945 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:59.092386 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:59.091997 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:59.104251 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:59.104224 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:30:59.436391 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:30:59.436310 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:33:15.526804 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:15.526773 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb"] Jul 20 16:33:25.778084 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.778002 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7"] Jul 20 16:33:25.781560 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.781542 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.783424 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.783406 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-329d3d\"/\"sub-test-model-329d3d-kserve-self-signed-certs\"" Jul 20 16:33:25.788670 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.788490 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7"] Jul 20 16:33:25.868488 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.868452 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmq7r\" (UniqueName: \"kubernetes.io/projected/b7c99e3f-0482-42ee-8b65-181abe8b2960-kube-api-access-hmq7r\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.868660 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.868507 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-home\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.868660 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.868583 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-model-cache\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.868660 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.868645 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-tmp-dir\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.868819 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.868662 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c99e3f-0482-42ee-8b65-181abe8b2960-tls-certs\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.868819 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.868751 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-dshm\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970175 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-tmp-dir\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970175 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c99e3f-0482-42ee-8b65-181abe8b2960-tls-certs\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970370 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970232 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-dshm\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970370 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970289 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmq7r\" (UniqueName: \"kubernetes.io/projected/b7c99e3f-0482-42ee-8b65-181abe8b2960-kube-api-access-hmq7r\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970458 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970408 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-home\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970505 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970459 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-model-cache\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970800 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970778 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-tmp-dir\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970905 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970839 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-model-cache\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.970905 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.970870 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-home\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.973156 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.973134 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-dshm\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.973361 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.973347 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c99e3f-0482-42ee-8b65-181abe8b2960-tls-certs\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:25.978082 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:25.978062 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmq7r\" (UniqueName: \"kubernetes.io/projected/b7c99e3f-0482-42ee-8b65-181abe8b2960-kube-api-access-hmq7r\") pod \"sub-test-model-329d3d-kserve-76f999d9d5-9trh7\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:26.091269 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:26.091234 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:26.228187 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:26.228158 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7"] Jul 20 16:33:26.230373 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:33:26.230347 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7c99e3f_0482_42ee_8b65_181abe8b2960.slice/crio-29fb3a381194875a17cf178c81f5fc3d59ab18adec6269c964e740a6b6277950 WatchSource:0}: Error finding container 29fb3a381194875a17cf178c81f5fc3d59ab18adec6269c964e740a6b6277950: Status 404 returned error can't find the container with id 29fb3a381194875a17cf178c81f5fc3d59ab18adec6269c964e740a6b6277950 Jul 20 16:33:27.168154 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:27.168118 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" event={"ID":"b7c99e3f-0482-42ee-8b65-181abe8b2960","Type":"ContainerStarted","Data":"5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6"} Jul 20 16:33:27.168535 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:27.168163 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" event={"ID":"b7c99e3f-0482-42ee-8b65-181abe8b2960","Type":"ContainerStarted","Data":"29fb3a381194875a17cf178c81f5fc3d59ab18adec6269c964e740a6b6277950"} Jul 20 16:33:27.184880 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:27.184834 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" podStartSLOduration=2.18481958 podStartE2EDuration="2.18481958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:33:27.182340249 +0000 UTC m=+2545.361017321" watchObservedRunningTime="2026-07-20 16:33:27.18481958 +0000 UTC m=+2545.363496652" Jul 20 16:33:30.547776 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.547735 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" podUID="a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" containerName="main" containerID="cri-o://823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8" gracePeriod=45 Jul 20 16:33:30.796600 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.796577 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:33:30.920297 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920216 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tls-certs\") pod \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " Jul 20 16:33:30.920297 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920260 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-dshm\") pod \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " Jul 20 16:33:30.920493 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920303 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q86t2\" (UniqueName: \"kubernetes.io/projected/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-kube-api-access-q86t2\") pod \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " Jul 20 16:33:30.920493 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920335 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-home\") pod \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " Jul 20 16:33:30.920493 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920359 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tmp-dir\") pod \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " Jul 20 16:33:30.920493 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920409 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-model-cache\") pod \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\" (UID: \"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb\") " Jul 20 16:33:30.920727 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920621 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-home" (OuterVolumeSpecName: "home") pod "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" (UID: "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:33:30.920727 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920641 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" (UID: "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:33:30.920841 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920764 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-model-cache" (OuterVolumeSpecName: "model-cache") pod "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" (UID: "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:33:30.920916 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920899 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-home\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:33:30.920973 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920923 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tmp-dir\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:33:30.920973 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.920938 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-model-cache\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:33:30.922690 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.922655 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-dshm" (OuterVolumeSpecName: "dshm") pod "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" (UID: "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:33:30.922786 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.922751 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" (UID: "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:33:30.922786 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:30.922778 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-kube-api-access-q86t2" (OuterVolumeSpecName: "kube-api-access-q86t2") pod "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" (UID: "a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb"). InnerVolumeSpecName "kube-api-access-q86t2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:33:31.022292 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.022268 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-tls-certs\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:33:31.022292 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.022288 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-dshm\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:33:31.022292 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.022297 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q86t2\" (UniqueName: \"kubernetes.io/projected/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb-kube-api-access-q86t2\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:33:31.189266 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.189174 2559 generic.go:356] "Generic (PLEG): container finished" podID="a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" containerID="823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8" exitCode=0 Jul 20 16:33:31.189266 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.189219 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" Jul 20 16:33:31.189490 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.189263 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" event={"ID":"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb","Type":"ContainerDied","Data":"823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8"} Jul 20 16:33:31.189490 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.189305 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb" event={"ID":"a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb","Type":"ContainerDied","Data":"28c1289cd5f24a9c51102924c7e3fc30ab47fd5631fedee863a6a8d22470ed09"} Jul 20 16:33:31.189490 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.189322 2559 scope.go:117] "RemoveContainer" containerID="823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8" Jul 20 16:33:31.198514 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.198498 2559 scope.go:117] "RemoveContainer" containerID="823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8" Jul 20 16:33:31.198788 ip-10-0-142-55 kubenswrapper[2559]: E0720 16:33:31.198770 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8\": container with ID starting with 823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8 not found: ID does not exist" containerID="823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8" Jul 20 16:33:31.198837 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.198797 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8"} err="failed to get container status \"823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8\": rpc error: code = NotFound desc = could not find container \"823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8\": container with ID starting with 823f7174656b91eaf15a4663b5efb1da3176c8f9f7663cbd42cd4d32304731a8 not found: ID does not exist" Jul 20 16:33:31.210003 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.209980 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb"] Jul 20 16:33:31.213826 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:31.213805 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/auth-test-model-329d3d-kserve-77ff778db7-4ktqb"] Jul 20 16:33:32.318295 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:32.318264 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" path="/var/lib/kubelet/pods/a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb/volumes" Jul 20 16:33:36.091761 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:36.091717 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:36.092213 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:36.091811 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:36.104624 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:36.104592 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:33:36.226962 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:33:36.226936 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:34:35.791373 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:35.791333 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7"] Jul 20 16:34:41.549164 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.549128 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk"] Jul 20 16:34:41.549605 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.549591 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" containerName="main" Jul 20 16:34:41.549652 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.549607 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" containerName="main" Jul 20 16:34:41.549714 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.549670 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5f79f37-6f6e-4c3f-adc9-4cf1d306a9cb" containerName="main" Jul 20 16:34:41.552972 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.552954 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.554949 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.554930 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-329d3d\"/\"rate-test-model-329d3d-kserve-self-signed-certs\"" Jul 20 16:34:41.562828 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.562779 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk"] Jul 20 16:34:41.718421 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.718382 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r85qp\" (UniqueName: \"kubernetes.io/projected/f0e3a09a-66bf-46cc-a100-aa32ca413bec-kube-api-access-r85qp\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.718572 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.718431 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tmp-dir\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.718572 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.718518 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-model-cache\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.718649 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.718570 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tls-certs\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.718649 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.718607 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-home\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.718742 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.718650 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-dshm\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.819418 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819309 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r85qp\" (UniqueName: \"kubernetes.io/projected/f0e3a09a-66bf-46cc-a100-aa32ca413bec-kube-api-access-r85qp\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.819418 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819357 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tmp-dir\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.819418 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819395 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-model-cache\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.819725 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819429 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tls-certs\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.819725 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819473 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-home\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.819839 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819754 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-dshm\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.819839 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819801 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tmp-dir\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.819984 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819847 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-home\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.820027 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.819996 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-model-cache\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.822274 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.822238 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-dshm\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.822413 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.822273 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tls-certs\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.828352 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.828331 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r85qp\" (UniqueName: \"kubernetes.io/projected/f0e3a09a-66bf-46cc-a100-aa32ca413bec-kube-api-access-r85qp\") pod \"rate-test-model-329d3d-kserve-9d7d8874c-bmjsk\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.863368 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.863343 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:41.999912 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:41.999883 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk"] Jul 20 16:34:42.003348 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:34:42.003322 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0e3a09a_66bf_46cc_a100_aa32ca413bec.slice/crio-70feae11a2991ac2c67be353a1ab96b2a002eb1250eda2171e1cc305ccdcafd0 WatchSource:0}: Error finding container 70feae11a2991ac2c67be353a1ab96b2a002eb1250eda2171e1cc305ccdcafd0: Status 404 returned error can't find the container with id 70feae11a2991ac2c67be353a1ab96b2a002eb1250eda2171e1cc305ccdcafd0 Jul 20 16:34:42.554902 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:42.554863 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" event={"ID":"f0e3a09a-66bf-46cc-a100-aa32ca413bec","Type":"ContainerStarted","Data":"bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f"} Jul 20 16:34:42.555296 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:42.554913 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" event={"ID":"f0e3a09a-66bf-46cc-a100-aa32ca413bec","Type":"ContainerStarted","Data":"70feae11a2991ac2c67be353a1ab96b2a002eb1250eda2171e1cc305ccdcafd0"} Jul 20 16:34:42.572163 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:42.572117 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" podStartSLOduration=1.572101476 podStartE2EDuration="1.572101476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:34:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:34:42.569931669 +0000 UTC m=+2620.748608743" watchObservedRunningTime="2026-07-20 16:34:42.572101476 +0000 UTC m=+2620.750778549" Jul 20 16:34:50.811551 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:50.811503 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" podUID="b7c99e3f-0482-42ee-8b65-181abe8b2960" containerName="main" containerID="cri-o://5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6" gracePeriod=45 Jul 20 16:34:51.064857 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.064797 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:34:51.115544 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115515 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmq7r\" (UniqueName: \"kubernetes.io/projected/b7c99e3f-0482-42ee-8b65-181abe8b2960-kube-api-access-hmq7r\") pod \"b7c99e3f-0482-42ee-8b65-181abe8b2960\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " Jul 20 16:34:51.115726 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115575 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-model-cache\") pod \"b7c99e3f-0482-42ee-8b65-181abe8b2960\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " Jul 20 16:34:51.115726 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115631 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-tmp-dir\") pod \"b7c99e3f-0482-42ee-8b65-181abe8b2960\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " Jul 20 16:34:51.115812 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115731 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-home\") pod \"b7c99e3f-0482-42ee-8b65-181abe8b2960\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " Jul 20 16:34:51.115812 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115773 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-dshm\") pod \"b7c99e3f-0482-42ee-8b65-181abe8b2960\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " Jul 20 16:34:51.115917 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115813 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c99e3f-0482-42ee-8b65-181abe8b2960-tls-certs\") pod \"b7c99e3f-0482-42ee-8b65-181abe8b2960\" (UID: \"b7c99e3f-0482-42ee-8b65-181abe8b2960\") " Jul 20 16:34:51.115917 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115892 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-model-cache" (OuterVolumeSpecName: "model-cache") pod "b7c99e3f-0482-42ee-8b65-181abe8b2960" (UID: "b7c99e3f-0482-42ee-8b65-181abe8b2960"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:34:51.116031 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115921 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b7c99e3f-0482-42ee-8b65-181abe8b2960" (UID: "b7c99e3f-0482-42ee-8b65-181abe8b2960"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:34:51.116031 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.115950 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-home" (OuterVolumeSpecName: "home") pod "b7c99e3f-0482-42ee-8b65-181abe8b2960" (UID: "b7c99e3f-0482-42ee-8b65-181abe8b2960"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:34:51.116139 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.116125 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-home\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:34:51.116180 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.116144 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-model-cache\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:34:51.116180 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.116159 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-tmp-dir\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:34:51.117791 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.117769 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c99e3f-0482-42ee-8b65-181abe8b2960-kube-api-access-hmq7r" (OuterVolumeSpecName: "kube-api-access-hmq7r") pod "b7c99e3f-0482-42ee-8b65-181abe8b2960" (UID: "b7c99e3f-0482-42ee-8b65-181abe8b2960"). InnerVolumeSpecName "kube-api-access-hmq7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:34:51.117937 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.117917 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-dshm" (OuterVolumeSpecName: "dshm") pod "b7c99e3f-0482-42ee-8b65-181abe8b2960" (UID: "b7c99e3f-0482-42ee-8b65-181abe8b2960"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:34:51.118086 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.118068 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c99e3f-0482-42ee-8b65-181abe8b2960-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7c99e3f-0482-42ee-8b65-181abe8b2960" (UID: "b7c99e3f-0482-42ee-8b65-181abe8b2960"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:34:51.217246 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.217209 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c99e3f-0482-42ee-8b65-181abe8b2960-dshm\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:34:51.217246 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.217237 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c99e3f-0482-42ee-8b65-181abe8b2960-tls-certs\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:34:51.217246 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.217249 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmq7r\" (UniqueName: \"kubernetes.io/projected/b7c99e3f-0482-42ee-8b65-181abe8b2960-kube-api-access-hmq7r\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:34:51.604033 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.603998 2559 generic.go:356] "Generic (PLEG): container finished" podID="b7c99e3f-0482-42ee-8b65-181abe8b2960" containerID="5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6" exitCode=0 Jul 20 16:34:51.604231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.604052 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" Jul 20 16:34:51.604231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.604065 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" event={"ID":"b7c99e3f-0482-42ee-8b65-181abe8b2960","Type":"ContainerDied","Data":"5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6"} Jul 20 16:34:51.604231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.604095 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7" event={"ID":"b7c99e3f-0482-42ee-8b65-181abe8b2960","Type":"ContainerDied","Data":"29fb3a381194875a17cf178c81f5fc3d59ab18adec6269c964e740a6b6277950"} Jul 20 16:34:51.604231 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.604111 2559 scope.go:117] "RemoveContainer" containerID="5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6" Jul 20 16:34:51.614665 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.614625 2559 scope.go:117] "RemoveContainer" containerID="5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6" Jul 20 16:34:51.615007 ip-10-0-142-55 kubenswrapper[2559]: E0720 16:34:51.614979 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6\": container with ID starting with 5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6 not found: ID does not exist" containerID="5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6" Jul 20 16:34:51.615101 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.615021 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6"} err="failed to get container status \"5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6\": rpc error: code = NotFound desc = could not find container \"5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6\": container with ID starting with 5f887737f92b11b1c0e32d0e3876fb132746b1e2bb7cf888961419ca127c64f6 not found: ID does not exist" Jul 20 16:34:51.625709 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.625665 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7"] Jul 20 16:34:51.631044 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.631019 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/sub-test-model-329d3d-kserve-76f999d9d5-9trh7"] Jul 20 16:34:51.864398 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.864309 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:51.864398 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.864375 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:51.877774 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:51.877744 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:34:52.318986 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:52.318953 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7c99e3f-0482-42ee-8b65-181abe8b2960" path="/var/lib/kubelet/pods/b7c99e3f-0482-42ee-8b65-181abe8b2960/volumes" Jul 20 16:34:52.622331 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:34:52.622250 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:35:53.556289 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:35:53.556204 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk"] Jul 20 16:36:08.577803 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.577756 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" podUID="f0e3a09a-66bf-46cc-a100-aa32ca413bec" containerName="main" containerID="cri-o://bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f" gracePeriod=45 Jul 20 16:36:08.833709 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.833619 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:36:08.875765 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.875740 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tmp-dir\") pod \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " Jul 20 16:36:08.875933 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.875780 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-home\") pod \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " Jul 20 16:36:08.875933 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.875801 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-model-cache\") pod \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " Jul 20 16:36:08.875933 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.875851 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r85qp\" (UniqueName: \"kubernetes.io/projected/f0e3a09a-66bf-46cc-a100-aa32ca413bec-kube-api-access-r85qp\") pod \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " Jul 20 16:36:08.875933 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.875877 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tls-certs\") pod \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " Jul 20 16:36:08.875933 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.875930 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-dshm\") pod \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\" (UID: \"f0e3a09a-66bf-46cc-a100-aa32ca413bec\") " Jul 20 16:36:08.876197 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.875981 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0e3a09a-66bf-46cc-a100-aa32ca413bec" (UID: "f0e3a09a-66bf-46cc-a100-aa32ca413bec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:36:08.876197 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.876022 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-home" (OuterVolumeSpecName: "home") pod "f0e3a09a-66bf-46cc-a100-aa32ca413bec" (UID: "f0e3a09a-66bf-46cc-a100-aa32ca413bec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:36:08.876197 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.876125 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0e3a09a-66bf-46cc-a100-aa32ca413bec" (UID: "f0e3a09a-66bf-46cc-a100-aa32ca413bec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:36:08.876369 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.876342 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tmp-dir\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:36:08.876369 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.876361 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-home\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:36:08.876462 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.876374 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-model-cache\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:36:08.878065 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.878036 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-dshm" (OuterVolumeSpecName: "dshm") pod "f0e3a09a-66bf-46cc-a100-aa32ca413bec" (UID: "f0e3a09a-66bf-46cc-a100-aa32ca413bec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:36:08.878615 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.878585 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0e3a09a-66bf-46cc-a100-aa32ca413bec" (UID: "f0e3a09a-66bf-46cc-a100-aa32ca413bec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:36:08.878615 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.878592 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e3a09a-66bf-46cc-a100-aa32ca413bec-kube-api-access-r85qp" (OuterVolumeSpecName: "kube-api-access-r85qp") pod "f0e3a09a-66bf-46cc-a100-aa32ca413bec" (UID: "f0e3a09a-66bf-46cc-a100-aa32ca413bec"). InnerVolumeSpecName "kube-api-access-r85qp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:36:08.977012 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.976983 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r85qp\" (UniqueName: \"kubernetes.io/projected/f0e3a09a-66bf-46cc-a100-aa32ca413bec-kube-api-access-r85qp\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:36:08.977012 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.977007 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e3a09a-66bf-46cc-a100-aa32ca413bec-tls-certs\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:36:08.977012 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.977018 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e3a09a-66bf-46cc-a100-aa32ca413bec-dshm\") on node \"ip-10-0-142-55.ec2.internal\" DevicePath \"\"" Jul 20 16:36:08.991321 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.991291 2559 generic.go:356] "Generic (PLEG): container finished" podID="f0e3a09a-66bf-46cc-a100-aa32ca413bec" containerID="bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f" exitCode=0 Jul 20 16:36:08.991457 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.991336 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" Jul 20 16:36:08.991457 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.991350 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" event={"ID":"f0e3a09a-66bf-46cc-a100-aa32ca413bec","Type":"ContainerDied","Data":"bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f"} Jul 20 16:36:08.991457 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.991381 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk" event={"ID":"f0e3a09a-66bf-46cc-a100-aa32ca413bec","Type":"ContainerDied","Data":"70feae11a2991ac2c67be353a1ab96b2a002eb1250eda2171e1cc305ccdcafd0"} Jul 20 16:36:08.991457 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:08.991397 2559 scope.go:117] "RemoveContainer" containerID="bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f" Jul 20 16:36:09.000643 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:09.000626 2559 scope.go:117] "RemoveContainer" containerID="bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f" Jul 20 16:36:09.000953 ip-10-0-142-55 kubenswrapper[2559]: E0720 16:36:09.000933 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f\": container with ID starting with bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f not found: ID does not exist" containerID="bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f" Jul 20 16:36:09.001056 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:09.000959 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f"} err="failed to get container status \"bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f\": rpc error: code = NotFound desc = could not find container \"bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f\": container with ID starting with bc375215cc93d1c1fbd22a73c9b03f5ad368d42a104f8a3e2b80192b7d02428f not found: ID does not exist" Jul 20 16:36:09.011172 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:09.011147 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk"] Jul 20 16:36:09.014942 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:09.014921 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-329d3d/rate-test-model-329d3d-kserve-9d7d8874c-bmjsk"] Jul 20 16:36:10.317947 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:36:10.317913 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e3a09a-66bf-46cc-a100-aa32ca413bec" path="/var/lib/kubelet/pods/f0e3a09a-66bf-46cc-a100-aa32ca413bec/volumes" Jul 20 16:37:04.005416 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:04.005388 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b76897bc6-st97x_361566c9-8d7d-4e58-9ac5-d128287f4adc/copy-manifests/0.log" Jul 20 16:37:04.011759 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:04.011740 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b76897bc6-st97x_361566c9-8d7d-4e58-9ac5-d128287f4adc/manager/0.log" Jul 20 16:37:04.450449 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:04.450417 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742720-t2xqz_10aba21d-f16a-4903-8086-20556f80ea4d/cleanup/0.log" Jul 20 16:37:06.177820 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:06.177796 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qsj2s_cc4f8f26-f770-43bb-b5e9-783cddb9f88f/manager/0.log" Jul 20 16:37:07.496456 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:07.496428 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bc6c54cc6-8zs5p_97d46b2f-d3d7-45b5-8ff2-5f9005cdb135/kube-auth-proxy/0.log" Jul 20 16:37:07.857066 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:07.857034 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-d9j42_fbbe3a31-924b-4e92-b0fc-948646b5336a/payload-pre-processing/0.log" Jul 20 16:37:08.450988 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:08.450963 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-j99rd_74751610-3411-43d1-ace7-1a964f330734/main/0.log" Jul 20 16:37:08.665404 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:08.665368 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-c68ss_1f165792-1f0b-4c1e-9a8a-715234e03d22/main/0.log" Jul 20 16:37:08.896793 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:08.896768 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rnl7c_4e6890d0-395b-4bac-acf6-478430768357/main/0.log" Jul 20 16:37:09.006338 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:09.006303 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-pnvf6_211e1746-4644-40fb-8abe-d5e1e10932b3/main/0.log" Jul 20 16:37:18.167336 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:18.167304 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kjkb7_c3d9ab51-3b57-4749-9de3-4724174739f4/global-pull-secret-syncer/0.log" Jul 20 16:37:18.397248 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:18.397220 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vqp9q_cfb70808-2749-4ee4-bc2d-df7d8a15f643/konnectivity-agent/0.log" Jul 20 16:37:18.503129 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:18.503056 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-55.ec2.internal_127f11aaeef3ff4f0262a9565d901f2f/haproxy/0.log" Jul 20 16:37:23.936702 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:23.936654 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qsj2s_cc4f8f26-f770-43bb-b5e9-783cddb9f88f/manager/0.log" Jul 20 16:37:26.053354 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.053331 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_830ea423-114e-4156-97c7-83aa8ba58190/alertmanager/0.log" Jul 20 16:37:26.119129 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.119099 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_830ea423-114e-4156-97c7-83aa8ba58190/config-reloader/0.log" Jul 20 16:37:26.167785 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.167758 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_830ea423-114e-4156-97c7-83aa8ba58190/kube-rbac-proxy-web/0.log" Jul 20 16:37:26.197364 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.197333 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_830ea423-114e-4156-97c7-83aa8ba58190/kube-rbac-proxy/0.log" Jul 20 16:37:26.229514 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.229488 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_830ea423-114e-4156-97c7-83aa8ba58190/kube-rbac-proxy-metric/0.log" Jul 20 16:37:26.255436 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.255417 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_830ea423-114e-4156-97c7-83aa8ba58190/prom-label-proxy/0.log" Jul 20 16:37:26.283867 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.283848 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_830ea423-114e-4156-97c7-83aa8ba58190/init-config-reloader/0.log" Jul 20 16:37:26.351860 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.351806 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-ztvqg_05f0fca8-f08b-4dba-96c4-0995069b8755/kube-state-metrics/0.log" Jul 20 16:37:26.370753 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.370731 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-ztvqg_05f0fca8-f08b-4dba-96c4-0995069b8755/kube-rbac-proxy-main/0.log" Jul 20 16:37:26.392026 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.392005 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-ztvqg_05f0fca8-f08b-4dba-96c4-0995069b8755/kube-rbac-proxy-self/0.log" Jul 20 16:37:26.420324 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.420307 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6dfd7cb58c-q2m5k_e744f9c8-1e9a-4690-a8e3-129270f32547/metrics-server/0.log" Jul 20 16:37:26.445463 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.445445 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-hc4wf_b8613015-77ce-4e72-9894-998c39717652/monitoring-plugin/0.log" Jul 20 16:37:26.483737 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.483715 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8spwx_86abd18a-786f-47f4-bf0a-3934f7125d77/node-exporter/0.log" Jul 20 16:37:26.509580 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.509564 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8spwx_86abd18a-786f-47f4-bf0a-3934f7125d77/kube-rbac-proxy/0.log" Jul 20 16:37:26.530707 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.530690 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8spwx_86abd18a-786f-47f4-bf0a-3934f7125d77/init-textfile/0.log" Jul 20 16:37:26.702750 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.702701 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-w6crt_151484f3-bc05-40c3-821c-8261ef3e5a3e/kube-rbac-proxy-main/0.log" Jul 20 16:37:26.725113 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.725090 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-w6crt_151484f3-bc05-40c3-821c-8261ef3e5a3e/kube-rbac-proxy-self/0.log" Jul 20 16:37:26.752993 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.752972 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-w6crt_151484f3-bc05-40c3-821c-8261ef3e5a3e/openshift-state-metrics/0.log" Jul 20 16:37:26.791089 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.791064 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9aff44ab-d6d2-4093-80af-58960f9647ab/prometheus/0.log" Jul 20 16:37:26.815666 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.815646 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9aff44ab-d6d2-4093-80af-58960f9647ab/config-reloader/0.log" Jul 20 16:37:26.837637 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.837619 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9aff44ab-d6d2-4093-80af-58960f9647ab/thanos-sidecar/0.log" Jul 20 16:37:26.865952 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.865933 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9aff44ab-d6d2-4093-80af-58960f9647ab/kube-rbac-proxy-web/0.log" Jul 20 16:37:26.887534 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.887514 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9aff44ab-d6d2-4093-80af-58960f9647ab/kube-rbac-proxy/0.log" Jul 20 16:37:26.908955 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.908939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9aff44ab-d6d2-4093-80af-58960f9647ab/kube-rbac-proxy-thanos/0.log" Jul 20 16:37:26.930440 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.930419 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9aff44ab-d6d2-4093-80af-58960f9647ab/init-config-reloader/0.log" Jul 20 16:37:26.959486 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.959434 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-fhzbr_a3f580e1-fd44-41dd-8eb0-124be7af2870/prometheus-operator/0.log" Jul 20 16:37:26.979315 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:26.979295 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-fhzbr_a3f580e1-fd44-41dd-8eb0-124be7af2870/kube-rbac-proxy/0.log" Jul 20 16:37:27.076287 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.076266 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b675c7cd7-shkfn_ec87b338-81e2-4a67-8265-217a7dc32157/telemeter-client/0.log" Jul 20 16:37:27.100947 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.100927 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b675c7cd7-shkfn_ec87b338-81e2-4a67-8265-217a7dc32157/reload/0.log" Jul 20 16:37:27.124081 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.124067 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b675c7cd7-shkfn_ec87b338-81e2-4a67-8265-217a7dc32157/kube-rbac-proxy/0.log" Jul 20 16:37:27.155746 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.155727 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d4478875-n2j2r_53570cf5-098f-44c5-875f-162878b01ed4/thanos-query/0.log" Jul 20 16:37:27.191725 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.191704 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d4478875-n2j2r_53570cf5-098f-44c5-875f-162878b01ed4/kube-rbac-proxy-web/0.log" Jul 20 16:37:27.214102 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.214051 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d4478875-n2j2r_53570cf5-098f-44c5-875f-162878b01ed4/kube-rbac-proxy/0.log" Jul 20 16:37:27.235748 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.235729 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d4478875-n2j2r_53570cf5-098f-44c5-875f-162878b01ed4/prom-label-proxy/0.log" Jul 20 16:37:27.259496 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.259478 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d4478875-n2j2r_53570cf5-098f-44c5-875f-162878b01ed4/kube-rbac-proxy-rules/0.log" Jul 20 16:37:27.285334 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:27.285318 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78d4478875-n2j2r_53570cf5-098f-44c5-875f-162878b01ed4/kube-rbac-proxy-metrics/0.log" Jul 20 16:37:28.429585 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.429554 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw"] Jul 20 16:37:28.429984 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.429969 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c99e3f-0482-42ee-8b65-181abe8b2960" containerName="main" Jul 20 16:37:28.430028 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.429986 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c99e3f-0482-42ee-8b65-181abe8b2960" containerName="main" Jul 20 16:37:28.430028 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.429996 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e3a09a-66bf-46cc-a100-aa32ca413bec" containerName="main" Jul 20 16:37:28.430028 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.430001 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e3a09a-66bf-46cc-a100-aa32ca413bec" containerName="main" Jul 20 16:37:28.430120 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.430063 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e3a09a-66bf-46cc-a100-aa32ca413bec" containerName="main" Jul 20 16:37:28.430120 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.430074 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7c99e3f-0482-42ee-8b65-181abe8b2960" containerName="main" Jul 20 16:37:28.433400 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.433381 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.435337 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.435317 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q8qg5\"/\"default-dockercfg-4bfpg\"" Jul 20 16:37:28.435459 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.435320 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q8qg5\"/\"kube-root-ca.crt\"" Jul 20 16:37:28.435755 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.435741 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q8qg5\"/\"openshift-service-ca.crt\"" Jul 20 16:37:28.447952 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.447932 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw"] Jul 20 16:37:28.522319 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.522297 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-sys\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.522459 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.522350 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-proc\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.522459 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.522410 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-podres\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.522555 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.522496 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlfqm\" (UniqueName: \"kubernetes.io/projected/7991dbc6-84d2-414e-af53-7ccbedf17276-kube-api-access-nlfqm\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.522555 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.522517 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-lib-modules\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.623766 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.623740 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlfqm\" (UniqueName: \"kubernetes.io/projected/7991dbc6-84d2-414e-af53-7ccbedf17276-kube-api-access-nlfqm\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.623889 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.623769 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-lib-modules\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.623889 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.623787 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-sys\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.623889 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.623838 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-proc\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.623889 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.623863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-podres\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.624094 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.623942 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-sys\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.624094 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.623944 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-proc\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.624094 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.623945 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-lib-modules\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.624094 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.624002 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7991dbc6-84d2-414e-af53-7ccbedf17276-podres\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.633052 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.633032 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlfqm\" (UniqueName: \"kubernetes.io/projected/7991dbc6-84d2-414e-af53-7ccbedf17276-kube-api-access-nlfqm\") pod \"perf-node-gather-daemonset-t2jnw\" (UID: \"7991dbc6-84d2-414e-af53-7ccbedf17276\") " pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.744042 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.743989 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:28.879978 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:28.879956 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw"] Jul 20 16:37:28.882292 ip-10-0-142-55 kubenswrapper[2559]: W0720 16:37:28.882255 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7991dbc6_84d2_414e_af53_7ccbedf17276.slice/crio-e1ee67f5721731d6fcbcf3d6d5409532d2b2c1e3fad0c047728baab54a79972c WatchSource:0}: Error finding container e1ee67f5721731d6fcbcf3d6d5409532d2b2c1e3fad0c047728baab54a79972c: Status 404 returned error can't find the container with id e1ee67f5721731d6fcbcf3d6d5409532d2b2c1e3fad0c047728baab54a79972c Jul 20 16:37:29.389016 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:29.388986 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" event={"ID":"7991dbc6-84d2-414e-af53-7ccbedf17276","Type":"ContainerStarted","Data":"093b6e802b57d53115c9dad2bd33a7e6431f9a8bfaae7e1a19ea99dee938b544"} Jul 20 16:37:29.389202 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:29.389037 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" event={"ID":"7991dbc6-84d2-414e-af53-7ccbedf17276","Type":"ContainerStarted","Data":"e1ee67f5721731d6fcbcf3d6d5409532d2b2c1e3fad0c047728baab54a79972c"} Jul 20 16:37:29.389202 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:29.389095 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:29.406298 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:29.406252 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" podStartSLOduration=1.406235146 podStartE2EDuration="1.406235146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:37:29.405018551 +0000 UTC m=+2787.583695625" watchObservedRunningTime="2026-07-20 16:37:29.406235146 +0000 UTC m=+2787.584912219" Jul 20 16:37:29.723957 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:29.723852 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7995cbcd6-s6j9x_346cca8c-b2d3-427a-afdb-255526d346fa/console/0.log" Jul 20 16:37:29.760574 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:29.760551 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-fnnpd_cbf42dd3-8ef3-48b3-86f7-5a66f14814dc/download-server/0.log" Jul 20 16:37:31.422602 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:31.422573 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9vmxs_31cf61c8-7ab9-4a0b-9995-be5987ffde36/dns/0.log" Jul 20 16:37:31.448973 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:31.448952 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9vmxs_31cf61c8-7ab9-4a0b-9995-be5987ffde36/kube-rbac-proxy/0.log" Jul 20 16:37:31.627115 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:31.627087 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qjhtn_ca77ff59-3d9a-4552-bb24-9b515e3adca5/dns-node-resolver/0.log" Jul 20 16:37:32.343265 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:32.343228 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8mc5h_37fdc62c-c7ed-46c0-a651-a2cc2a420cb0/node-ca/0.log" Jul 20 16:37:33.845271 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:33.845243 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bc6c54cc6-8zs5p_97d46b2f-d3d7-45b5-8ff2-5f9005cdb135/kube-auth-proxy/0.log" Jul 20 16:37:33.987692 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:33.987623 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-d9j42_fbbe3a31-924b-4e92-b0fc-948646b5336a/payload-pre-processing/0.log" Jul 20 16:37:34.897824 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:34.897793 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ktzcc_1e0170e8-2d46-490e-aecf-ccccc6eb7b73/serve-healthcheck-canary/0.log" Jul 20 16:37:35.404244 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:35.404216 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q8qg5/perf-node-gather-daemonset-t2jnw" Jul 20 16:37:35.775983 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:35.775870 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-frsgc_f32c9452-5c16-43fd-8612-4c19214966b9/kube-rbac-proxy/0.log" Jul 20 16:37:35.829185 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:35.829155 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-frsgc_f32c9452-5c16-43fd-8612-4c19214966b9/exporter/0.log" Jul 20 16:37:35.870245 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:35.870205 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-frsgc_f32c9452-5c16-43fd-8612-4c19214966b9/extractor/0.log" Jul 20 16:37:39.357176 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:39.357151 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bc6c54cc6-8zs5p_97d46b2f-d3d7-45b5-8ff2-5f9005cdb135/kube-auth-proxy/0.log" Jul 20 16:37:39.445288 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:39.445254 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-d9j42_fbbe3a31-924b-4e92-b0fc-948646b5336a/payload-pre-processing/0.log" Jul 20 16:37:39.446384 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:39.446367 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b76897bc6-st97x_361566c9-8d7d-4e58-9ac5-d128287f4adc/manager/0.log" Jul 20 16:37:39.482774 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:39.482666 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b76897bc6-st97x_361566c9-8d7d-4e58-9ac5-d128287f4adc/copy-manifests/0.log" Jul 20 16:37:40.977395 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:40.977359 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8c9c5dcb7-rdc5t_bf7bf3a7-18b1-4ebf-8764-68cf6e88b43e/manager/0.log" Jul 20 16:37:41.016415 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:41.016386 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-qqvf9_4b5e6cdc-d6e2-460f-9329-6d5618ef1aed/openshift-lws-operator/0.log" Jul 20 16:37:41.027255 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:41.027231 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bc6c54cc6-8zs5p_97d46b2f-d3d7-45b5-8ff2-5f9005cdb135/kube-auth-proxy/0.log" Jul 20 16:37:41.133012 ip-10-0-142-55 kubenswrapper[2559]: I0720 16:37:41.132981 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-d9j42_fbbe3a31-924b-4e92-b0fc-948646b5336a/payload-pre-processing/0.log"